Method and apparatus for authenticating area biometric scanners

ABSTRACT

Methods and apparatuses for authenticating a biometric scanner, such as area type finger print scanners, involves estimating unique intrinsic characteristics of the scanner (scanner pattern), that are permanent over time, and can identify a scanner even among scanners of the same manufacturer and model. Image processing and analysis are used to extract a scanner pattern from images acquired with the scanner. The scanner pattern is used to verify whether the scanner that acquired a particular image is the same as the scanner that acquired one or several images during enrollment of the biometric information. Authenticating the scanner can prevent subsequent security attacks using counterfeit biometric information on the scanner, or on the user authentication system.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of application Ser. No. 13/480,122,filed May 24, 2012 and claims the benefit of the U.S. provisional patentapplication No. 61/489,607 filed on May 24, 2011, which are incorporatedherein by reference in their entirety. The United States utility patentapplication Ser. No. 12/838,952 filed on Jul. 19, 2010, entitled “Methodand Apparatus for Authenticating Area Biometric Scanners” is alsoincorporated herein by reference in its entirety.

GOVERNMENT SUPPORT

The subject matter disclosed herein was made with partial governmentfunding and support under MURI W911-NF-0710287 awarded by the UnitedStates Army Research Office (ARO). The government has certain rights inthis invention.

FIELD OF TECHNOLOGY

The exemplary implementations described herein relate to securitysystems and methods, and more particularly, to methods forauthentication of area and swipe biometric scanners.

BACKGROUND

Authentication verifies the claim about the identity of an entity (e.g.,a person or a system). The information about human physiological andbehavioral traits, collectively called biometric information or simplybiometrics, can be used to identify a particular individual with a highdegree of certainty and therefore can authenticate this individual bymeasuring, analyzing, and using these traits. Well-known types ofbiometrics include face photographs, fingerprints, iris and retinascans, palm prints, and blood vessel scans. A great variety of specificdevices, hereinafter referred to as biometric scanners, are used tocapture and collect biometric information, and transform the biometricinformation into signals for further use and processing. Despite alladvantages (e.g., convenience) of using biometrics over using othermethods for authentication of people, the biometric information can havesignificant weaknesses. For example, the biometric information has a lowlevel of secrecy because it can be captured surreptitiously by anunintended recipient and without the consent of the person whom itbelongs to. Furthermore, if once compromised, the biometric informationis not easily changeable or replaceable, and it cannot be revoked.Another problem is that the biometric information is inexact, may changeover time, and is “noisy” (e.g., it is not like a password or a PINcode) as it cannot be reproduced exactly from one measurement toanother, and therefore it can be matched only approximately, which givesrise to authentication errors. All these weaknesses and problems imperilthe confidence in the reliable use of biometrics in everyday life.

One of the most widely used biometrics is the fingerprint. It has beenused for identifying individuals for over a century. The surface of theskin of a human fingertip consists of a series of ridges and valleysthat form a unique fingerprint pattern. The fingerprint patterns arehighly distinct, they develop early in life, and their details arerelatively permanent over time. In the last several decades, theextensive research in algorithms for identification based on fingerprintpatterns has led to the development of automated biometric systems usingfingerprints for various applications, including law enforcement, bordercontrol, enterprise access, and access to computers and to otherportable devices. Although fingerprint patterns change little over time,changes in the environment (e.g., humidity and temperature changes),cuts and bruises, dryness and moisture of the skin, and changes due toaging pose certain challenges for the identification of individuals byusing fingerprint patterns in conjunction with scanners. However,similar problems also exist when identifying individuals by using otherbiometric information.

Using biometric information for identifying individuals typicallyinvolves two steps: biometric enrolment and biometric verification. Forexample, in case of fingerprints, a typical biometric enrolment requiresacquiring one or more (typically three) fingerprint images with afingerprint scanner, extracting from the fingerprint image informationthat is sufficient to identify the user, and storing the extractedinformation as template biometric information for future comparison withsubsequently acquired fingerprint images. A typical biometricverification involves acquiring another, subsequent image of thefingertip and extracting from that image query biometric informationwhich is then compared with the template biometric information. If thetwo pieces of information are sufficiently similar, the result is deemedto be a biometric match. In this case, the user's identity is verifiedpositively and the user is authenticated successfully. If the comparedinformation is not sufficiently similar, the result is deemed abiometric non-match, the verification of the user's identity isnegative, and the biometric authentication fails.

One proposed way for improving or enhancing the security of the systemsthat use biometric information is by using digitalwatermarking—embedding information into digital signals that can beused, for example, to identify the signal owner or to detect tamperingwith the signal. The digital watermark can be embedded in the signaldomain, in a transform domain, or added as a separate signal. If theembedded information is unique for every particular originator (e.g., incase of image, the originator is the camera or the scanner used toacquire the image), the digital watermark can be used to establish theauthenticity of the digital signal by using methods taught in the priorart. However, robust digital watermarking, i.e., one that cannot beeasily detected, removed, or copied, requires computational power thatis typically not available in biometric scanners and, generally, comesat high additional cost. In order to ensure the uniqueness of theoriginator (e.g., the camera or the scanner), the originator also needsan intrinsic source of randomness.

To solve the problem of associating a unique number with a particularsystem or a device, it has been proposed to store this number in a flashmemory or in a mask Read Only Memory (ROM). The major disadvantages ofthis proposal are the relatively high added cost, the man-maderandomness of the number, which number is usually generated duringdevice manufacturing, and the ability to record and track this number bythird parties. Prior art also teaches methods that introduce randomnessby exploiting the variability and randomness created by mismatch andother physical phenomena in electronic devices or by using physicallyunclonable functions (PUF) that contain physical components with sourcesof randomness. Such randomness can be explicitly introduced (as a designby the system designer) or intrinsically present (e.g., signalpropagation delays within batches of integrated circuits are naturallydifferent). However, all of these proposed methods and systems come atadditional design, manufacturing, and/or material cost.

Prior art teaches methods for identification of digital cameras based onthe two types of sensor pattern noise: fixed pattern noise andphoto-response non-uniformity. However, these methods are not suited tobe used for biometric authentication using fingerprints because saidmethods require many (in the order of tens to one hundred) images. Theseprior art methods also use computationally intensive signal processingwith many underlying assumptions about the statistical properties of thesensor pattern noise. Attempts to apply these methods for authenticationof optical fingerprint scanners have been made in laboratory studieswithout any real success and they are insufficiently precise whenapplied to capacitive fingerprint scanners, because the methodsimplicitly assume acquisition models that are specific for the digitalcameras but which models are very different from the acquisition processof capacitive fingerprint scanners. The attempts to apply these methodsto capacitive fingerprint scanners only demonstrated theirunsuitability, in particular for systems with limited computationalpower. In addition, these methods are not suited for a big class offingerprint scanners known as swipe (also slide or sweep) fingerprintscanners, in which a row (or a column) of sensing elements sequentially,row by row (or column by column), scan the fingertip skin, from whichscans a fingerprint image is then constructed. The acquisition processin digital cameras is inherently different as cameras typically acquirethe light coming from the object at once, e.g., as a “snapshot,” so thateach sensing element produces the value of one pixel in the image, not awhole row (or column) of pixels as the swipe scanners do. Prior art alsoteaches methods for distinguishing among different types and models ofdigital cameras based on their processing artifacts (e.g., their colorfilter array interpolation algorithms), which is suited for cameraclassification (i.e., determining the brand or model of a given camera),but not for camera identification (i.e., determining which particularcamera has acquired a given image).

Aside from the high cost associated with the above described securityproposals, another disadvantage is that they cannot be used in biometricscanners that have already been manufactured and placed into service.

SUMMARY

In order to overcome security problems associated with biometricscanners and systems in the prior art, exemplary illustrativenon-limiting implementations of methods and apparatuses are hereindescribed which enhance or improve the security of existing or newlymanufactured biometric scanners and systems by authenticating thebiometric scanner itself in addition to authenticating the submittedbiometric information.

A biometric scanner converts the biometric information into signals thatare used by a system, e.g., a computer, a smart phone, or a door lock,to automatically verify the identity of a person. A fingerprint scanner,a type of biometric scanner, converts the surface or subsurface of theskin of a fingertip into one or several images. In practice, thisconversion process can never be made perfect. The imperfections inducedby the conversion process can be classified into two general categories:imperfections that are largely time invariant, hereinafter referred toas scanner pattern, and imperfections that change over time, hereinafterreferred to as scanner noise. As will be described herein, the scannerpattern is unique to a particular scanner and, therefore, it can be usedto verify the identity of the scanner; this process hereinafter isreferred to as scanner authentication.

By requiring authentication of both the biometric information and thebiometric scanner, the submission of counterfeit images—obtained byusing a different biometric scanner or copied by other means and thenreplayed—can be detected, thereby preventing authentication of thesubmitted counterfeit biometric images. In this way, attacks on thebiometric scanner or on the system that uses the biometric informationcan be prevented, improving the overall security of the biometricauthentication.

The illustrative non-limiting implementations disclosed herein aredirected to methods that estimate the scanner pattern of a fingerprintscanner without violating the integrity of the scanner by disassemblingit, performing measurements inside of it, or applying any otherintrusive methods. The scanner pattern is estimated solely from an imageor from several images that are acquired by the scanner. This estimatedscanner pattern is used for scanner authentication.

The scanner authentication comprises (1) a scanner enrolment, e.g.,estimating from a digital image and then storing the scanner pattern ofa legitimate, authentic scanner, and (2) a scanner verification, e.g.,extracting the scanner pattern from a digital image and comparing itwith the stored scanner pattern to verify if the digital image has beenacquired with the authentic fingerprint scanner or not. As will beappreciated by those skilled in the art, the scanner authentication willprovide an increased level of security of the biometric authentication.For example, the scanner authentication can detect attacks on thefingerprint scanner, such as detecting an image containing thefingerprint pattern of the legitimate user and acquired with theauthentic fingerprint scanner that has been replaced by another imagethat still contains the fingerprint pattern of the legitimate user buthas been acquired with another, unauthentic fingerprint scanner. Thistype of attack has become an important security threat as the widespreaduse of biometric technologies makes the biometric informationessentially publicly available.

The herein described illustrative non-limiting implementations ofscanner authentication can be used in any system that authenticatesusers based on biometric information, especially in systems that operatein uncontrolled (i.e., without human supervision) environments, inparticular in portable devices, such as PDAs, cellular phones, smartphones, multimedia phones, wireless handheld devices, and generally anymobile devices, including laptops, notebooks, netbooks, etc., becausethese devices can be easily stolen, giving to an attacker physicalaccess to them and thus the opportunity to interfere with theinformation flow between the biometric scanner and the system. Thegeneral but not limited areas of application of the exemplaryillustrative non-limiting implementations described herein are in bankapplications, mobile commerce, for access to health care anywhere and atany time, for access to medical records, etc. The subject matterdescribed herein can also be used in hardware tokens. For example, thesecurity of a hardware token equipped with a fingerprint scanner (i.e.,a token using fingerprint authentication instead of authentication basedon a secret code) can be improved by adding the above described scannerauthentication.

In one exemplary implementation of the herein described subject matter,a machine-implemented method for estimating the scanner pattern of abiometric scanner comprises processing at least one digital image,produced by the biometric scanner. The scanner pattern is estimated fromthis at least one digital image by using a wavelet decomposition andreconstruction, masking as useful at most all pixels by comparing theirmagnitude with a predetermined threshold, and storing these usefulpixels for future use. This scanner pattern can be used to identify thescanner.

In another exemplary implementation of the herein described subjectmatter, a machine-implemented method for authenticating biometricscanners comprises processing at least one digital image, produced bythe biometric scanner. The biometric scanner is first enrolled byestimating the scanner pattern from this at least one digital image byusing wavelet decomposition and reconstruction, masking as useful atmost all pixels by comparing their magnitude with a predeterminedthreshold, and then storing these useful pixels as a template scannerpattern for future comparison. The biometric scanner is verified bysubsequently processing at least one digital image and processing themto estimate a query scanner pattern by using a wavelet decomposition andreconstruction and masking as useful at most all pixels by comparingtheir magnitude with a predetermined value, and storing these usefulpixels as a query scanner pattern. The template scanner pattern and thequery scanner pattern are then matched and a decision is made as towhether the images have been acquired with the same biometric scanner orwith different biometric scanners.

In another exemplary implementation of the herein described subjectmatter, a machine-implemented method for estimating the scanner patternof a biometric scanner comprises processing at least one digital image,produced by the biometric scanner. The scanner pattern is then estimatedfrom this at least one digital image by averaging pixels of the image tocompute a line (vector) of pixels, filtering this line (vector) ofpixels, and storing this line (vector) of pixels for future use. Thisscanner pattern can be used to identify the scanner.

In another exemplary implementation of the herein described subjectmatter, a machine-implemented method for authenticating biometricscanners comprises processing at least one digital image, produced bythe biometric scanner. The biometric scanner is first enrolled byestimating the scanner pattern from this at least one digital image byaveraging pixels of the image to compute a line (vector) of pixels,filtering this line (vector) of pixels, and storing this line (vector)of pixels as a template scanner pattern for future comparison. Thebiometric scanner is verified by subsequently processing at least onedigital image to estimate a query scanner pattern by averaging pixels ofthe image to compute a line (vector) of pixels, filtering this line(vector) of pixels, and storing this line (vector) of pixels as a queryscanner pattern. The template scanner pattern and the query scannerpattern are then matched and a decision is made as to whether the imageshave been acquired with the same biometric scanner or with differentbiometric scanners.

The above described methods can be implemented by an electronicprocessing circuit configured to perform the enumerated processes oroperations. Suitable electronic processing circuits for performing thesemethods include processors including at least one CPU and associatedinputs and outputs, memory devices, and accessible programmedinstructions for carrying out the methods, programmable gate arraysprogrammed to carry out the methods, and ASICs specially designedhardware devices for carrying out the methods.

BRIEF DESCRIPTION OF THE DRAWINGS

These and further aspects of the exemplary illustrative non-limitingimplementations will be better understood in light of the followingdetailed description of illustrative exemplary non-limitingimplementations in conjunction with the drawings, of which:

FIG. 1 is a block diagram of a fingerprint scanner;

FIG. 2 is a block diagram of a fingerprint scanner connected over anetwork to a system that uses the image acquired with the fingerprintscanner;

FIG. 3 is a block diagram of a fingerprint scanner connected directly toa system that uses the image acquired with the fingerprint scanner;

FIG. 4 is a block diagram of a fingerprint scanner that is part of asystem that uses the image acquired with the fingerprint scanner;

FIG. 5 is an exemplary block diagram of a system that uses biometricinformation;

FIG. 6 shows an example of scanner imperfections;

FIG. 7 shows columns of pixels from two images: one acquired with airand another one acquired with a fingertip applied to the scanner platen;

FIG. 8 is a conceptual signal flow diagram of operation of the signalprocessing modules of one of the methods disclosed in the presentinvention;

FIG. 9 is a conceptual signal flow diagram of operation of the signalprocessing modules of another method disclosed in the present invention;

FIG. 10 is a flow diagram of the signal processing steps of oneexemplary implementation;

FIG. 11 is a flow diagram of the signal processing steps of anotherexemplary implementation;

FIG. 12 is an exemplary flow diagram of the method for bipartiteenrolment according to one exemplary implementation;

FIG. 13 is an exemplary flow diagram of the method for bipartiteverification according to one exemplary implementation;

FIG. 14 is a flow diagram of the method for bipartite verificationaccording to another exemplary implementation;

FIG. 15 is a table with exemplary implementations of the method forbipartite authentication depending on the object used for scannerenrolment and for scanner verification and the corresponding levels ofsecurity each implementation provides;

FIG. 16 shows the scanner authentication decisions in one exemplaryimplementation of the 2D Wavelet Method which employs the correlationcoefficient as a similarity score;

FIG. 17 shows the input signal and the output signal of an one exemplaryimplementation of the Filtering Module of the Averaging Method;

FIG. 18 shows the scanner authentication decisions in one exemplaryimplementation of the of the Averaging Method which employs thecorrelation coefficient as a similarity score; and

FIG. 19 shows an exemplary illustrative architecture of a single-leveltwo-dimensional discrete wavelet decomposition and reconstruction.

DETAILED DESCRIPTION

A typical fingerprint scanner, shown as block 110 in FIG. 1 generallycomprises a fingerprint sensor 112, which reads the fingerprint pattern,a signal processing unit 114, which processes the reading of the sensorand converts it into an image, and an interface unit 116, whichtransfers the image to system (not shown in the figure) that uses it.The system that uses the image includes, but is not limited to, adesktop or server computer, a door lock for access control, a portableor mobile device such as a laptop, PDA or cellular telephone, hardwaretoken, or any other access control device.

As shown in FIG. 2, the fingerprint scanner 110 can be connected to thesystem 130 that uses the image via wireless or wired links and a network120. The network 120 can be, for example, the Internet, a wireless“Wi-Fi” network, a cellular telephone network, a local area network, awide area network, or any other network capable of communicatinginformation between devices. As shown in FIG. 3, the fingerprint scanner110 can be directly connected to the system 132 that uses the image. Asshown in FIG. 4, the fingerprint scanner 110 can be an integral part ofthe system 134 that uses the image.

Nevertheless, in any of the cases shown in FIGS. 2-4, an attacker whohas physical access to the system can interfere with the informationflow between the fingerprint scanner and the system in order toinfluence the operation of the authentication algorithms that arerunning on the system, for example, by replacing the image that isacquired with the fingerprint scanner by another image that has beenacquired with another fingerprint scanner or by an image that has beenmaliciously altered (e.g., tampered with).

The system 130 in FIG. 2, the system 132 in FIG. 3, and the system 134in FIG. 4, may have Trusted Computing (TC) functionality; for example,the systems may be equipped with a Trusted Platform Module (TPM) thatcan provide complete control over the software that is running and thatcan be run in these systems. Thus, once the image, acquired with thefingerprint scanner, is transferred to the system software for furtherprocessing, the possibilities for an attacker to interfere andmaliciously modify the operation of this processing become very limited.However, even in a system with such enhanced security, an attacker whohas physical access to the system can still launch an attack byreplacing the image acquired with a legitimate, authentic fingerprintscanner, with another digital image. For example, an attacker who hasobtained an image of the fingerprint of the legitimate user can initiatean authentication session with the attacker's own fingerprint and then,at the interface between the fingerprint scanner and the system, theattacker can replace the image of attacker's fingerprint with the imageof the fingerprint of the legitimate user. Most authenticationalgorithms today will not detect this attack but will report that theuser authentication to the system is successful.

FIG. 5 illustrates a typical system 200 that uses biometric informationand used to implement the methods and apparatuses disclosed herein caninclude one or more processors 202, which comprise but are not limitedto general-purpose microprocessors (including CISC and RISCarchitectures), signal processors, microcontrollers, or other types ofprocessors executing instructions, with their associated inputs andoutputs. The system 200 may also have a read-only memory (ROM) 204,which includes but is not limited to PROM, EPROM, EEPROM, flash memory,or any other type of memory used to store computer instructions anddata. The system 200 may further have random-access memory (RAM) 206,which includes but is not limited to SRAM, DRAM, DDR, or any othermemory used to store computer instructions and data. The system 200 canalso have electronic processing circuit and digital hardware 208, whichincludes but is not limited to programmable field arrays with suitableprogramming by blown fuses, field-programmable gate arrays (FPGA),complex programmable logic devices (CPLD), programmable logic arrays(PLA), programmable array logic (PAL), application-specific integratedcircuits (ASIC), designed and fabricated to perform specific functions,or any other type of hardware that can perform computations and processsignals. The system 200 may further have one or several input/outputinterfaces (I/O) 210, which include but are not limited to a keypad, akeyboard, a touchpad, a mouse, speakers, a microphone, one or severaldisplays, USB interfaces, interfaces to one or more biometric scanners,digital cameras, or any other interfaces to peripheral devices. Thesystem 200 may also have one or several communication interfaces 212that connect the system to wired networks, including but not limited toEthernet or fiber-optical links, and wireless networks, including butnot limited to CDMA, GSM, WiFi, GPRS, WiMAX, IMT-2000, 3GPP, or LTE. Thesystem 200 may also have storage devices (not shown), including but notlimited to hard disk drives, optical drives (e.g., CD and DVD drives),or floppy disk drives. The system 200 may also have TC functionality;for example, it may be equipped with a TPM that can provide completecontrol over the software that is running and that can be run in it.

Today, many low-cost and small-size live-scan fingerprint scanners areavailable and used in various biometric systems. Depending on thesensing technology and the type of the sensor used for imageacquisition, fingerprint scanners fall into one of the three generalcategories: optical, solid-state (e.g., capacitive, thermal, based onelectric field, and piezo-electric), and ultrasound. Anotherclassification of fingerprint scanners is based on the method ofapplying the fingertip to the scanner. In the first group, referred toas touch or area fingerprint scanners, the fingertip is applied to thesensor and then the corresponding digital image is acquired withoutrelative movement of the fingertip over the sensor, taking a “snapshot”of the fingertip skin. This is simple but has several disadvantages: thesensor may become dirty, a latent fingerprint may remain on the surfacethat may impede the subsequent image acquisition, and there are alsohygienic concerns. Furthermore, the size of the sensor area (which islarge) is directly related to the cost of the scanner.

In the second group, referred to as swipe, sweep, or slide fingerprintscanners, after applying the fingertip to the scanner, the fingertip ismoved over the sensor so that the fingertip skin is scannedsequentially, row by row (or column by column), and then the signalprocessing unit constructs an image of the fingerprint pattern from thescanned rows (or columns). Swiping overcomes the major disadvantages ofthe touching mode and can significantly reduce the cost as the sensorcan have height of only several pixels. The swipe fingerprint scannersare particularly suited for portable devices because of their small sizeand low cost.

Fingerprint scanners essentially convert the biometric information,i.e., the surface or subsurface of the skin of a fingertip, into one orseveral images. In practice, this conversion process can never be madeperfect. The imperfections induced by the conversion process can beclassified into two general categories: (a) imperfections that arepersistent and largely do not change over time, which are hereinafterreferred to as scanner pattern, and (b) imperfections that changerapidly over time, which are hereinafter referred to as scanner noise.

The scanner pattern can be a function of many and diverse factors in thescanner hardware and software, e.g., the specific sensing method, theused semiconductor technology, the chip layout, the circuit design, andthe post-processing. Furthermore, pinpointing the exact factors, muchless quantifying them, can be difficult because such informationtypically is proprietary. Nevertheless, our general observation is thatthe scanner pattern stems from the intrinsic characteristics of theconversion hardware and software and is mainly caused by non-idealitiesand variability in the fingerprint sensor; however, the signalprocessing unit and even the interface unit (see FIG. 1) can alsocontribute to it. The intrinsic characteristics that cause the scannerpattern remain relatively unchanged over time. Variations in theseintrinsic characteristics, however, may still exist and may be caused byenvironmental changes such as changes in the temperature, air pressure,and air humidity, and sensor surface moisture; material aging;scratches, liquid permeability, and ESD impact on the sensor surface,changes in the illumination (for optical scanners); etc. The scannernoise is generally caused by non-idealities in the conversion processthat vary considerably within short periods of time. Typical examplesfor scanner noise are the thermal noise, which is inherently present inany electronic circuit, and the quantization noise, e.g., the signaldistortion introduced in the conversion of an analog signal into adigital signal. An example for the combined effect of such imperfections(i.e., the scanner pattern and the scanner noise) is shown in FIG. 6.The image 300, shown on the left side of FIG. 6, is an image acquiredwith no object applied to the scanner platen. A small rectangular blockof pixels from the image 300 is enlarged and shown on the right side ofFIG. 6 as block 302. The three adjacent pixels 304, 306, and 308 ofblock 302 have different scales of the gray color: pixel 304 is darkerthan pixel 308 and pixel 306 is brighter than pixel 308.

Generally, the scanner pattern of a fingerprint scanner can be estimatedfrom two types of images depending on the type of the object applied tothe fingerprint scanner:

-   -   1. A predetermined, known a priori, object. Since the object is        known, the differences (in the general sense, not limited only        to subtraction) between the image acquired with the        predetermined object and the theoretical image that would be        acquired if the fingerprint scanner were ideal reveal the        scanner pattern because the image does not contain a fingerprint        pattern.    -   2. A fingertip of a person that, generally, is not known a        priori. The acquired image in this case is a composition of the        fingerprint pattern, the scanner pattern, and the scanner noise.

The scanner pattern is a sufficiently unique, persistent, andunalterable intrinsic characteristic of the fingerprint scanners even tothose of exactly the same technology, manufacturer, and model. Themethods and apparatuses disclosed herein are able to distinguish thepattern of one scanner from the pattern of another scanner of exactlythe same model by estimating the pattern from a single image, acquiredwith each scanner. In this way, the scanner pattern can be used toenhance the security of a biometric system by authenticating thescanner, used to acquire a particular fingerprint image, and thus detectattacks on the scanner, such as detecting an image containing thefingerprint pattern of the legitimate user and acquired with theauthentic fingerprint scanner replaced by another image that stillcontains the fingerprint pattern of the legitimate user but has beenacquired with another, unauthentic fingerprint scanner. The scannerpattern can also be used by itself as a source of randomness, unique foreach scanner (and also for the system if the scanner is an integral partof it), that identifies the scanner, in other security applications,both already present today and in the future.

The process of matching involves the comparison of a sample of animportant feature from an object under test (also known as query) with astored template of the same feature representing its normalrepresentation (also known as enrolled feature). One or more images canbe used to generate the sample under test, using different methods.Similarly, one or more images can be used to generate the normalrepresentation using different methods. The performance of the match (asmeasured for example by scores) will depend on the number of images usedfor the sample, the number of images used to generate the normalrepresentation, the signal processing and the methods used for matching.These should be selected carefully as the best combination will varydepending on the device and the signal processing methodology used. Forexample for the wavelet methods disclosed herein, it is better toaverage scores, while for the averaging methods, it is better to averagethe scanner pattern estimates. In the context and applications of thepresent invention, it is possible to have the following sets of imagesthat are being matched:

1. One enrolled image and one query image. A similarity score iscomputed between the scanner pattern of the enrolled image and the queryimage, which score is then compared with a threshold.

2. Many enrolled images and one query image. In this case, the matchingcan be done in two ways:

-   -   (a) The similarity scores for the scanner patterns of each pair        {enrolled image, query image} are computed and then these scores        are averaged to produce a final similarity score, which average        is then compared with a threshold to make a decision;    -   (b) The scanner patterns of the enrolled images are computed and        these scanner patterns are averaged to compute an average        scanner pattern, which is then used to compute a similarity        score with the scanner pattern of the query image. The resulting        score is compared with a threshold.

3. Many enrolled images and many query images. Four cases can bedefined:

-   -   (a) The similarity scores for the scanner patterns of each pair        (enrolled image, query image) are computed and then these scores        are averaged to produce a final score, which is compared with a        threshold.    -   (b) The scanner patterns of the enrolled images are computed and        then they are averaged to compute an average scanner pattern,        which is then used to compute a similarity score with the        scanner pattern of each pair (average scanner of the enrolled        images, the scanner pattern of a query image). The resulting        scores are then averaged, and this final score is compared with        a threshold.    -   (c) The scanner patterns of the query images are computed and        then they are averaged to compute an average scanner pattern,        which is then used to compute a similarity score with the        scanner pattern of each pair {scanner pattern of an enrolled        image, average scanner pattern of the query images}. The        resulting scores are then averaged, and this final score is        compared with a threshold.    -   (d) The average scanner pattern of the enrolled images is        computed and the average scanner pattern of the query images is        computed. Then a similarity score between the two average        patterns is computed, and this final score is compared with a        threshold. The performance in this case, however, when masking        is used, may be suboptimal because the number of common pixels        in the two average patterns may be small.

D.1 Signal Models

The actual function describing the relationship among the scannerpattern, the scanner noise, and the fingerprint pattern (when present)can be very complex. This function depends on the particular fingerprintsensing technology and on the particular fingerprint scanner design andimplementation, which are usually proprietary. Furthermore, even if theexact function is known or determined, using it for estimating thescanner pattern may prove difficult, mathematically intractable, orrequire computationally intensive and extensive signal processing.However, this function can be simplified into a composition ofadditive/subtractive terms, multiplicative/dividing terms, andcombinations of them by taking into account only the major contributingfactors and by using approximations. This simple, approximate model ofthe actual function is henceforth referred to as the “signal model.”

In developing signal models for capacitive fingerprint scanners, we usedreadily available commercial devices sold by AuthenTec, Inc. (Melbourne,Fla., USA) and Verdicom, Inc. (now defunct). Both the area and the swipecapacitive fingerprint scanners of AuthenTec used herein have beendeveloped by UPEK, Inc. (formerly from Emeryville, Calif., USA, now partof AuthenTec); after the merger of UPEK with AuthenTec in 2010, thesecapacitive fingerprint scanners were integrated into the product line ofAuthenTec. The technology of the capacitive fingerprint scanners ofVeridicom have been acquired and later scanners manufactured and sold byFujitsu (Tokyo, Japan).

When the image, acquired with the fingerprint scanner, is not compressedor further enhanced by image processing algorithms to facilitate thebiometric authentication, or is compressed or enhanced but the scannerpattern information contained in it is not substantially altered, thepixel values g(i,j) of the image (as saved in a computer file) at rowindex i and column index j can be expressed as one of the two models:

a) Signal Model A:

$\begin{matrix}{{g\left( {i,j} \right)} = {\frac{s\left( {i,j} \right)}{1 + {{s\left( {i,j} \right)}{f\left( {i,j} \right)}}} + {n\left( {i,j,t} \right)}}} & (1)\end{matrix}$

b) Signal Model B:

$\begin{matrix}{{g\left( {i,j} \right)} = {\frac{s\left( {i,j} \right)}{1 + {f\left( {i,j} \right)}} + {n\left( {i,j,t} \right)}}} & (2)\end{matrix}$

where ƒ(i,j) is the fingerprint pattern, s(i,j) is the scanner pattern,and n(i,j,t) is the scanner noise, which also depends on the time tbecause the scanner noise is time varying (by definition). Alloperations in Equations (1) and (2), i.e., the addition, themultiplication, and the division, are element by element (i.e., pixel bypixel) because the Point Spread Function of these fingerprint scanners,viewed as a two-dimensional linear space-invariant system, can be wellapproximated with a Dirac delta function. Signal Model A is bettersuited for the capacitive fingerprint scanners of UPEK/AuthenTec, whileSignal Model B is better suited for the capacitive fingerprint scannersof Veridicom/Fujitsu. The typical range of g(i,j) is from 0 to 255grayscale levels (8 bits/pixel), although some scanner implementationsmay produce narrower range of values and thus make estimating thescanner pattern less accurate. Furthermore, some scanners may produceimages with spatial resolution that is different from their nativespatial resolution (i.e., the resolution determined by the distancebetween the sensing elements and used to acquire the image), forexample, by interpolating between the pixel values of the image toproduce pixel values corresponding to a different spatial resolution.These examples of signal processing may significantly alter the scannerpattern in the image and/or make its estimation particularly difficult.

D.1.1 Signal Characteristics

D.1.1.1 Scanner Noise

Henceforth the term scanner noise refers to the combined effect oftime-varying factors that result in short-term variations, i.e., fromwithin several seconds to much faster, in the pixel values ofconsecutively acquired images under the same acquisition conditions(e.g., when the fingertip applied to the scanner is not changed inposition, the force with which the fingertip is pressed to the scannerplaten is kept constant, and the skin moisture is unchanged) and theunder exactly the same environmental conditions (e.g., without changesin the temperature, air humidity, or air pressure). Examples for factorscontributing to the scanner noise are the thermal, shot, flicker, and soon noises that are is present in any electronic circuit, and thequantization noise, which is the distortion introduced by the conversionof an analog signal into a digital signal. Other contributing noisesources may also exist. A plausible assumption is that the combinedeffect of all such factors is similar to the combined effect of manynoise sources, which is modeled as a (temporal additive) noise in, forexample, communication systems. Therefore, of importance are thestatistical characteristics only of this aggregation of all short-termtemporal noise sources.

Viewed as a one-dimensional signal represented as a function of time t(i.e., as its temporal characteristics) at a given pixel, the scannernoise can be approximated as having a Gaussian probability distributionwith zero mean N(0,σ_(n) ²). This Gaussian model can also be used toapproximate the scanner noise across the scanner platen (e.g., alongcolumns or rows) at a given time t, i.e., the scanner noise spatialcharacteristics. The variance of the scanner noise may vary across thescanner platen of one scanner, may vary across different scanners evenof the same model and manufacturer, and may vary with the environmentalconditions (especially with the temperature). We estimated that thescanner noise variance σ_(n) ², both in time and in space, on averagecan be approximately assumed about 1.77 for Scanner Model A and about0.88 for Scanner Model B. Deviations from the Gaussian distribution ofthe amplitude probability distribution of the scanner noise, such asoutliers, heavy tails, burstiness, and effects due to the coarsequantization, may also be present, requiring robust signal processingalgorithms as disclosed herein.

D.1.1.2 Scanner Pattern

Because of the presence of scanner noise, which is time varying, it isonly possible to estimate the scanner pattern. The scanner pattern,viewed as a two-dimensional random process, i.e., a random processdependent on two independent spatial variables, can be well approximatedby a Gaussian random field, i.e., a two-dimensional random variable thathas a Gaussian distribution N(μ_(s),σ_(s) ²), where μ_(s) is the meanand σ_(s) ² is the variance of the scanner pattern. This random field isnot necessarily stationary in the mean, i.e., the mean μ_(s) may varyacross one and the same image (e.g., as a gradient effect). The scannerpattern may also change roughly uniformly (e.g., with an approximatelyconstant offset) for many pixels across the scanner platen among imagesacquired with the same fingerprint scanner under different environmentalconditions, e.g., under different temperatures or different moistures(i.e., water). Because of the variable mean μ_(s) and this (nonconstant)offset, the absolute value of the scanner pattern may create problemsfor the signal processing. We incorporate these effects in the followingmodel of the scanner pattern s(i,j) as a sum of two components:

s(i,j)=μ_(s)(i,j)+s _(ν)(i,j).  (3)

The first component, μ_(s)(i,j), is essentially the mean of the scannerpattern. It slowly varies in space (i.e., across the scanner platen) butmay (considerably) change over time in the long term and also underdifferent environmental conditions and other factors. As it is notreliably reproducible, it is difficult to be used as a persistentcharacteristic of each scanner, and therefore it needs to be removedfrom consideration; its effect can be mitigated and in certain casescompletely eliminated. The second component, s_(ν)(i,j), rapidly variesin space but is relatively invariant in time (in both the short and thelong term) and under different environmental conditions. This variablepart s_(ν)(i,j) of the scanner pattern mean does not changesignificantly and is relatively stable under different conditions. It issufficiently reproducible and can serve as a persistent characteristicof the scanner. Furthermore, it determines the variance σ_(s) ², which,therefore, is relatively constant. This type of permanence of thescanner pattern is a key element of the exemplary implementations. Inaddition to the variable mean and offset, however, the scanner patterns(i,j) may also exhibit other deviations from the theoretical Gaussiandistribution, such as outliers and heavy tails. A specific peculiaritythat can also be attributed as scanner pattern are malfunctioning (i.e.,“dead”) pixels that produce constant pixel values regardless of theobject applied to the scanner platen at their location; the pixel valuesthey produce, however, may also change erratically. All these effectsrequired choosing robust signal processing as disclosed herein.

For area scanners, the scanner pattern s(i,j) depends on two parameters(i and j) which are the row index and column index, respectively. Forswipe scanners, a line, being a row or a column, of sensor elementsperforms an instant scan of a tiny area of the fingertip skin andconverts the readings into a line of pixels. In case when a row ofsensor elements scans the fingertip, since the row is only one, thescanner pattern for all rows (i.e., along columns) is the same, i.e.,s(i,j)=s(j) for all i. Similarly, when a column of sensor elements scansthe fingertip, the scanner pattern for all columns (i.e., along rows) isthe same, i.e., s(i,j)=s(i) for all j. Although the methodology forestimating the scanner pattern and its parameters as disclosed herein isspecified for area scanners, its application to swipe scanners becomesstraightforward by using these simplifications.

The mean μ_(s) and the variance σ_(s) ² of the scanner pattern arecritical parameters, and they can be determined in two ways:

-   -   One or both of the parameters are computed before the        fingerprint scanner is placed into service, i.e., they are        predetermined and fixed to typical values that are computed by        analyzing a single image or a plurality of images of the same        fingerprint scanner or a batch of scanners of the same type and        model;    -   One or both of the parameters are computed dynamically during        the normal operation of the fingerprint scanner by analyzing a        single image or a plurality of images of the same fingerprint        scanner. Since these computed values will be closer to the        actual, true values of the parameters for a particular        fingerprint scanner, the overall performance may be higher than        in the first method. However, this increase in the performance        may come at higher computational cost and weaker security.        Nevertheless, either way of computing either one of the        parameters, i.e., μ_(s) or σ_(s) ², leads to computing estimates        of the actual, true values of these parameters because of two        reasons: (i) the scanner pattern itself is a random field for        which only a finite amount of data, i.e., image pixels, is        available, and (ii) there is presence of scanner noise, which is        a time-varying random process.

D.1.1.2.1 Scanner Pattern Mean and Variance

Estimates of the scanner pattern mean and scanner pattern variance canbe computed from a single image or from a plurality of images acquiredwith a predetermined object applied to the scanner platen. The preferredpredetermined object for Signal Model A and Signal Model B is air, i.e.,no object is applied to the scanner platen, but other predeterminedobjects are also possible, e.g., water. When the object ispredetermined, i.e., not a fingertip of a person, and is air, thenƒ(i,j)=0. Furthermore, for either one of the signal models, the pixelvalue at row index i and column index j of an image acquired with apredetermined object applied to the scanner is:

g ^((po))(i,j)=s(i,j)+n(i,j,t).  (4)

Averaging many pixel values g^((po))(i,j) acquired sequentially with oneand the same fingerprint scanner will provide the best estimate of thescanner pattern s(i,j), because the average over time of the scannernoise n(i,j,t) at each pixel will tend to 0 (subject to the law of largenumbers), as with respect to time, the scanner noise is a zero-meanrandom process. Thus, if g_(k) ^((po))(i,j) is the pixel value of thek-th image acquired with a particular fingerprint scanner, then theestimate of the scanner pattern Ŝ(i,j) at row index i and column index jis:

$\begin{matrix}{{\hat{s}\left( {i,j} \right)} = {\frac{1}{K}{\sum\limits_{k = 1}^{K}{g_{k}^{({po})}\left( {i,j} \right)}}}} & (5)\end{matrix}$

where K is the number of images used for averaging (K can be as small asten). Then, an estimate s of the mean μ_(s) of the scanner pattern canbe computed using the formula for the sample mean:

$\begin{matrix}{\overset{\_}{s} = {\frac{1}{I \cdot J}{\sum\limits_{i = 1}^{I}{\sum\limits_{j = 1}^{J}{\hat{s}\left( {i,j} \right)}}}}} & (6)\end{matrix}$

where I is the total number of rows and J is the total number of columnsin the image. The estimate {circumflex over (σ)}_(s) ² of the scannerpattern variance σ_(s) ² can then be computed using:

$\begin{matrix}{{\hat{\sigma}}_{s}^{2} = {\frac{1}{I \cdot J}{\sum\limits_{i = 1}^{I}{\sum\limits_{j = 1}^{J}{\left( {{\hat{s}\left( {i,j} \right)} - \overset{\_}{s}} \right)^{2}.}}}}} & (7)\end{matrix}$

Instead of the biased estimate in Equation (7), it is also possible tocompute the unbiased estimate by dividing by (I−1)(J−1) instead of by(I,J) as in Equation (7):

$\begin{matrix}{{\hat{\sigma}}_{s}^{2} = {\frac{1}{\left( {I - 1} \right) \cdot \left( {J - 1} \right)}{\sum\limits_{i = 1}^{I}{\sum\limits_{j = 1}^{J}{\left( {{\hat{s}\left( {i,j} \right)} - \overset{\_}{s}} \right)^{2}.}}}}} & (8)\end{matrix}$

However, since the mean μ_(s) of the scanner pattern is not constant andalso depends on the temperature and the moisture, using s in thecomputation of the estimate of the scanner pattern variance may besuboptimal. Therefore, it is better to compute the local estimates{circumflex over (μ)}_(s)(i,j) of the sample mean of the scanner patternfor each pixel at row index i and column index j by averaging the pixelvalues in blocks of pixels:

$\begin{matrix}{{{\hat{\mu}}_{s}\left( {i,j} \right)} = {\frac{1}{L \cdot R}{\sum\limits_{l = {- {\lfloor\frac{L}{2}\rfloor}}}^{\lfloor\frac{L - 1}{2}\rfloor}{\sum\limits_{r = {- {\lfloor\frac{R}{2}\rfloor}}}^{\lfloor\frac{R - 1}{2}\rfloor}{\hat{s}\left( {{i + l},{j + r}} \right)}}}}} & (9)\end{matrix}$

where the integers L and R define the dimensions of the block over whichthe local estimate is computed and └.┘ is the floor function (e.g.,└2.6┘=2 and └−1.4┘=−2). Setting L and R in the range from about 5 toabout 20 yields the best performance, but using values outside thisrange is also possible. When the index (i+1) or the index (j+r) fallsoutside the image boundaries, the size of the block is reduced toaccommodate the block size reduction in the particular computation;i.e., fewer pixels are used in the sums in Equation (9) for averaging tocompute the corresponding local estimate {circumflex over (μ)}_(s)(i,j).

In another exemplary implementation, the local estimate {circumflex over(μ)}_(s)(i,j) of the sample mean μ_(s) is computed in a single dimensioninstead of in two dimensions (i.e., in blocks) as in Equation (9). Thiscan be done along rows, along columns, or along any one-dimensionalcross section of the image. For example, computing the local estimate{circumflex over (μ)}_(s)(i,j) for each pixel with row index i andcolumn index j can be done by averaging the neighboring pixels in thecolumn j, reducing Equation (9) to:

$\begin{matrix}{{{\hat{\mu}}_{s}\left( {i,j} \right)} = {\frac{1}{L}{\sum\limits_{i = {- {\lfloor\frac{L}{2}\rfloor}}}^{\lfloor\frac{L - 1}{2}\rfloor}{{\hat{s}\left( {{i + l},j} \right)}.}}}} & (10)\end{matrix}$

When the index (i+1) in Equation (10) falls outside the imageboundaries, i.e., for the pixels close to the image edges, the number ofpixels L used for averaging in Equation (10) is reduced to accommodatethe block size reduction in the computation of the corresponding localestimate {circumflex over (μ)}_(s)(i,j).

Finally, the estimate {circumflex over (σ)}_(s) ² of the scanner patternvariance σ_(s) ² of can be computed over all pixels in the image usingthe following Equation (11):

$\begin{matrix}{{\hat{\sigma}}_{s}^{2} = {\frac{1}{I \cdot J}{\sum\limits_{i = 1}^{I}{\sum\limits_{j = 1}^{J}{\left( {{\hat{s}\left( {i,j} \right)} - {{\hat{\mu}}_{s}\left( {i,j} \right)}} \right)^{2}.}}}}} & (11)\end{matrix}$

Instead of the biased estimate of the variance of the scanner pattern asgiven in Equation (11), it is also possible to compute an unbiasedestimate by dividing by (I−1)(J−1) instead of by (I,J) as in Equation(11):

$\begin{matrix}{{\hat{\sigma}}_{s}^{2} = {\frac{1}{\left( {I - 1} \right) \cdot \left( {J - 1} \right)}{\sum\limits_{i = 1}^{I}{\sum\limits_{j = 1}^{J}{\left( {{\hat{s}\left( {i,j} \right)} - {{\hat{\mu}}_{s}\left( {i,j} \right)}} \right)^{2}.}}}}} & (12)\end{matrix}$

In another exemplary implementation, the estimate {circumflex over(σ)}_(s) ² of the scanner pattern variance σ_(s) ² can be computed usingonly some of the pixels in the image, not all pixels as in Equation (11)or Equation (12). For example, the averaging in these equations can bedone only over a block of pixels, only along predetermined rows, onlyalong predetermined columns, only along predetermined cross-sections ofthe image, or any combinations of these. This approach may decrease theaccuracy of the estimate {circumflex over (σ)}_(x) ², but it may bebeneficial because it will reduce the requirements for computationalpower.

The estimate {circumflex over (σ)}_(s) ² of the scanner pattern variancedepends on the particular fingerprint scanner and the particular signalmodel. It also depends on the method for computing the estimate becausedue to the specifics of the sensing process in each scanner type and theparticular hardware and software design of each scanner type and model,the estimates {circumflex over (σ)}_(x) ² for one and the same scannerwhen computed over blocks of pixels, over columns of pixels, and overrows of pixels may differ from one another. To compute the local meanestimate {circumflex over (μ)}_(s)(i,j), we used a 11-tap moving-filteralong columns, i.e., L=11 in Equation (10). Using it, for Signal ModelA, we found that the estimate {circumflex over (σ)}_(x) ² typicallyfalls in the range from about 12 to about 20, and for Signal Model Bfrom about 7 to about 17.

The estimate s of the mean μ_(s) of the scanner pattern and the localmean estimate {circumflex over (μ)}_(s)(i,j) may depend on theparticular scanner and the conditions under which they are estimated(e.g., temperature and moisture). The local mean estimate {circumflexover (μ)}_(s)(i,j) also changes across the image area, i.e., it is notconstant in function of i and j. The estimates s and {circumflex over(μ)}_(s)(i,j) also depend on the signal model, and for Signal Model Athey are in the range from about 150 to about 220 grayscale levels andfor Signal Model B in the range from about 200 to about 250 grayscalelevels.

A disadvantage of this approach for estimating the scanner patternvariance is that it requires additional images acquired with apredetermined object applied to the scanner platen. This may requirethat during the scanner enrolment two groups of images be acquired: onegroup with a predetermined object applied to the scanner and anothergroup with user's fingerprint applied to the scanner. This may increasethe computational time and may weaken the security as the legitimatescanner may be replaced between the acquisitions of the two groups ofimages. About 10 images acquired with a predetermined object applied tothe scanner pattern are sufficient to yield an accurate estimate of thescanner pattern variance.

The above described exemplary implementations for estimating the scannerpattern mean and variance can be done either during the scannerenrolment or prior to the scanner enrolment.

In one exemplary implementation, the estimate of the scanner patternvariance in case when it is computed and set before a particularfingerprint scanner is placed into service is computed by averaging theestimates of the scanner pattern variances of a batch of M scanners.Thus, if {circumflex over (σ)}_(s,m) ² is the estimate of the scannerpattern variance of the m-th scanner in the batch, then the averageestimate of the scanner pattern variance is:

$\begin{matrix}{{\hat{\sigma}}_{s}^{2} = {\frac{1}{M}{\sum\limits_{m = 1}^{M}{{\hat{\sigma}}_{s,m}^{2}.}}}} & (13)\end{matrix}$

We used Equation (13) and determined that a sufficiently accurateestimate {circumflex over (σ)}_(s) ² of the scanner pattern variance forSignal Model A is about 15.5 (with a batch of 22 scanners) and forSignal Model B is about 8.6.

Computing and using an accurate estimate of the scanner pattern varianceis important, but the methods disclosed in the exemplary implementationsare sufficiently robust against wide deviations of the estimate{circumflex over (σ)}_(s) ² from the true value of the scanner patternvariance {circumflex over (σ)}_(s) ², i.e., the overall performanceremains relatively unchanged. Furthermore, separating the variable partof the scanner pattern from the scanner noise is virtually impossiblebecause both of them are approximately Gaussian processes and for themethods disclosed herein of importance is the variance of the scannerpattern and the scanner noise. Our analysis showed that this combinedvariance is approximately equal to the sum of {circumflex over (σ)}_(s)² and {circumflex over (σ)}_(n) ², and for Signal Model A, it is fromabout 13.5 to about 21, with an average about 17, and for Signal ModelB—about 9.5 on average.

D.1.1.2.2 Scanner Pattern Spatial Dependence

The random field of the scanner pattern s(i,j) can be approximatelymodeled as white, i.e., its one-dimensional and its two-dimensionalautocorrelation functions can be approximated by a Dirac delta function,one-dimensional and two-dimensional, respectively. The accuracy of thisapproximation depends on the particular signal model. This accuracy mayalso be different along the two main axes of the two-dimensionalautocorrelation function due to the specifics in the hardware andsoftware implementations of the particular fingerprint scanner type andmodel, including but not limited to its sensing technology and theaddressing of its sensing elements. For example, for Signal Model A, thetwo-dimensional autocorrelation function along its column axis istypically closer to the Dirac delta function than it is along its rowaxis, and although it does exhibit some correlation, this correlation islimited and justifies the assumption for being largely uncorrelated. Thetwo-dimensional autocorrelation function of Signal Model B along itscolumn axis is as close to the Dirac delta function as it is along itsrow axis, although it exhibits non-negligible oscillations along bothaxes with frequency very close to it radians, which is a noticeabledeparture from the Dirac delta function and implies correlation withinthe scanner pattern along columns and rows.

D.1.1.3 Fingerprint Pattern

Henceforth we refer to the two-dimensional function ƒ(i,j) as introducedin Equation (1) for Signal Model A and in Equation (2) for Signal ModelB as a fingerprint pattern.

The surface of the fingertip skin (as well as its subsurface) is asequence of ridges and valleys. This surface is read by the fingerprintscanner and represented as a two-dimensional signal via the imageacquisition. Along with other imperfections introduced by thefingerprint scanner in this representation, the acquisition process mayalso include nonlinear transformations, such as projection of thethree-dimensional fingertip onto the two-dimensional scanner platen anda sensing process that reads the ridges and valleys and converts thesereadings into electrical signals, which signals are further processedand converted into a digital image. As result of such nonlineartransformations, the fingerprint pattern ƒ(i,j) may become a nonlinearfunction of the actual surface (and/or subsurface) of the fingertipskin.

For our purposes, the fingerprint pattern ƒ(i,j), in each of its twodimensions, can be roughly viewed as one dominant, single-frequencyoscillation along with its harmonics. The frequency of this oscillationdepends on the width of the ridges and valleys, which are specific foreach individual. This frequency also depends on the particular type offinger (e.g., thumbs typically have wider ridges and deeper valleys thanlittle fingers have, on the hands of one and the same person). Alsotypically, index fingers have narrower ridges and valleys than thumbs,and wider than little fingers. This frequency also depends on the gender(male fingers typically have wider ridges and valleys than femalefingers have) and on the age (adults usually have wider ridges andvalleys than children have). Finally, this frequency may even varywithin one and the same fingerprint. As a very approximate model, afrequency of about 0.63 radians per pixel is sufficiently representativefor modeling purposes in the context of the exemplary implementations.

FIG. 7 is a representative figure depicting the pixel values (ingrayscale levels) of a column of pixels from two images: one imageacquired with no object applied to the scanner platen (i.e., with air),shown in dotted gray curve, and another image acquired with a fingertipapplied to the scanner platen, shown in solid black curve. The regionsthat correspond to valleys and to ridges in the image acquired with afingertip applied to the scanner platen are also shown. One importantobservation in this figure is that the pixel values in certain ridgesare constant (equal to grayscale level 1), i.e., the scanner becomessaturated and the signal “clips.” We observed that this saturation canbe due to several reasons: (a) strong pressure of the fingertip to thescanner platen (which is typical for thumb fingers as they arestronger), (b) wide ridges (also typical for thumbs and to some extentfor the middle fingers as they are naturally larger than the otherfingers), and (c) highly moisturized fingers. While this saturation ofthe signal to 1 is not a problem for the fingerprint recognition,estimating the scanner pattern from such saturated regions is notpossible and, therefore, it is very important that such regions areexcluded from further processing.

The range of the fingerprint pattern ƒ(i,j), as it is introduced inEquations (1) and (2), is (0, 1]. The pixel values g(i,j) at row index iand column index j for the two general regions of the fingertip skin,ridges and valleys, taking into account that s(i,j)>>1 for either signalmodel, are approximately as follows:

-   -   In the regions with ridges, ƒ(i,j) is close to 1. Hence:        -   for Signal Model A:

${g^{(r)}\left( {i,j} \right)} \approx {\frac{s\left( {i,j} \right)}{1 + {s\left( {i,j} \right)}} + {n\left( {i,j,t} \right)}} \approx {1 + {n\left( {i,j,t} \right)}}$

-   -   -   for Signal Model B:

${g^{(r)}\left( {i,j} \right)} \approx {\frac{s\left( {i,j} \right)}{1 + 1} + {n\left( {i,j,t} \right)}} \approx {\frac{s\left( {i,j} \right)}{2} + {n\left( {i,j,t} \right)}}$

-   -   In the regions with valleys, ƒ(i,j) is close to 0. Hence:        -   for Signal Model A:

${g^{(v)}\left( {i,j} \right)} \approx {\frac{s\left( {i,j} \right)}{1 + 0} + {n\left( {i,j,t} \right)}} \approx {{s\left( {i,j} \right)} + {n\left( {i,j,t} \right)}}$

-   -   -   for Signal Model B:

${g^{(v)}\left( {i,j} \right)} \approx {\frac{s\left( {i,j} \right)}{1 + 0} + {n\left( {i,j,t} \right)}} \approx {{s\left( {i,j} \right)} + {n\left( {i,j,t} \right)}}$

Therefore, in the regions with valleys, for either one of the signalmodels (see Equation (4)):

g ^((ν))(i,j)≈s(i,j)+n(i,j,t)=g ^((po))(i,j).  (14)

D.2 Signal Inversion

Equations (1) and (2) model the relationship between the scanner patterns(i,j) and the fingerprint pattern ƒ(i,j). Because of the division andthe multiplication operations in them, directly separating s(i,j) fromƒ(i,j) is difficult. In order to facilitate this separation, the pixelvalues g(i,j) in the image can be inverted. Thus, for every row index iand column index j, we define h(i,j):

$\begin{matrix}{{h\left( {i,j} \right)}\overset{\Delta}{=}\left\{ \begin{matrix}\frac{1}{g\left( {i,j} \right)} & {{{for}\mspace{14mu} {g\left( {i,j} \right)}} \neq 0} \\1 & {{{for}\mspace{14mu} {g\left( {i,j} \right)}} = 0}\end{matrix} \right.} & (15)\end{matrix}$

This inversion applied to Signal Model A transforms differently therelationship between the scanner pattern and the fingerprint patternfrom the same inversion applied to Signal Model B, but the end result ofthe inversion for the regions with valleys is very similar for the twosignal models because (a) the scanner noise n(i,j,t) is much weaker asignal than the scanner pattern s(i,j) and (b) the value of the scannerpattern ƒ(i,j) in the regions with valleys is close to 0. Hence, forSignal Model A:

$\begin{matrix}\begin{matrix}{{h\left( {i,j} \right)} = \frac{1}{\frac{s\left( {i,j} \right)}{1 + {{s\left( {i,j} \right)} \cdot {f\left( {i,j} \right)}}} + {n\left( {i,j,t} \right)}}} \\{\approx {\frac{1}{s\left( {i,j} \right)} + {f\left( {i,j} \right)}}}\end{matrix} & (16)\end{matrix}$

and for Signal Model B:

$\begin{matrix}\begin{matrix}{{h\left( {i,j} \right)} = \frac{1}{\frac{s\left( {i,j} \right)}{1 + {f\left( {i,j} \right)}} + {n\left( {i,j,t} \right)}}} \\{\approx {\frac{1}{s\left( {i,j} \right)} + {\frac{f\left( {i,j} \right)}{s\left( {i,j} \right)}.}}}\end{matrix} & (17)\end{matrix}$

Since the mean μ_(s) (which in this case is the local mean μ_(s)(i,j))of the scanner pattern s(i,j) is much larger than its standard deviationσ_(s) and the mean μ_(s) only varies slowly, the variations of thescanner pattern s(i,j) about its local mean are small. Therefore, theterm

$\frac{f\left( {i,j} \right)}{s\left( {i,j} \right)}$

in Equation (17) is approximately equal to

$\frac{f\left( {i,j} \right)}{\mu_{s}},$

which implies that locally, the fingerprint pattern ƒ(i,j) essentiallyis simply scaled down by an almost constant factor μ_(s), but itswaveform shape as such is preserved; we refer to this scaled downversion of the fingerprint pattern as ƒ′(i,j). Hence, using thisapproximation, Equation (17) becomes similar to Equation (16) and it is:

$\begin{matrix}{{h\left( {i,j} \right)} \approx {\frac{1}{s\left( {i,j} \right)} + \frac{f\left( {i,j} \right)}{\mu_{s}}} \approx {\frac{1}{s\left( {i,j} \right)} + {{f^{\prime}\left( {i,j} \right)}.}}} & (18)\end{matrix}$

Because of its importance in the analysis that follows, we also definet(i,j) as the inverse of the scanner pattern:

$\begin{matrix}{{t\left( {i,j} \right)}\overset{\Delta}{=}{\frac{1}{s\left( {i,j} \right)}.}} & (19)\end{matrix}$

Thus, by applying the signal inversion of Equation (15), themultiplicative relationship between the scanner pattern s(i,j) and thefingerprint pattern ƒ(i,j) in Equations (1) and (2) is transformed intoa sum of two terms, one of which represents the scanner pattern and theother one—the fingerprint pattern:

h(i,j)≈t(i,j)+ƒ(i,j).  (20)

This makes their separation possible using simple signal processing. Inaddition, we developed a Gaussian approximation for the inverse of aGaussian random variable, according to which t(i,j) has approximately aGaussian distribution N(μ_(i), σ_(i) ²) with:

$\begin{matrix}{{\mu_{t} = {{\frac{1}{\mu_{s}}\mspace{14mu} {and}\mspace{20mu} \sigma_{t}^{2}} = \frac{\sigma_{s}^{2}}{\mu_{s}^{4}}}},} & (21)\end{matrix}$

where μ_(s) is the mean and the σ_(s) ² is the variance of the scannerpattern s(i,j). This approximation is sufficiently accurate whenμ_(s)>100 and μ_(s)>>σ_(s), both of which hold true for both signalmodels. Using the range of values for the scanner pattern mean p, andstandard deviation o, for Signal Model A, μ_(t) falls in the range fromabout 4.54·10⁻³ to about 6.67·10⁻³ with a typical value of about5.0·10⁻³, and σ_(t) falls in the range from about 0.72·10⁻⁴ to about1.99·10⁻⁴ with a typical value of about 0.98·10⁻⁴. For Signal Model B,μ_(t) falls in the range from about 4.0·10⁻³ to about 5.0·10⁻³ with atypical value of about 4.54·10⁻³, and σ_(t) falls in the range fromabout 0.42·10⁻⁴ to about 1.03·10⁻⁴ with typical value of about0.61·10⁻⁴. Note: because of the inevitable presence of scanner noise,which we have neglected in the current discussion about the inversion,of importance for the signal processing of the pixels in the valleys isnot the variance σ_(s) ² of the scanner pattern alone, but the varianceof the combined scanner pattern and spatial scanner noise. Since thescanner pattern and the spatial scanner noise can both be assumed to beapproximately Gaussian, their sum is also a Gaussian, and, therefore,the inversion approximation is still applicable, but has to be used withthe sum of their variances, i.e., σ_(s) ²+σ_(n) ².

In summary, the problem of separating the scanner pattern and thefingerprint pattern, which are in a complex relationship with eachother, is thus reduced to separating a Gaussian signal from an additiveand roughly sinusoidal signal, which can be done in a straight forwardand computationally-efficient way. Because of this simplification, thesignal modeling in Equations (1) and (2) and the signal inversion inEquation (15) are key elements of the exemplary implementationsdescribed herein. Two downsides of this inversion, however, are that (a)the inversion may require care in implementing it in digitalprecision-limited systems, e.g., with fixed-point arithmetic, because ofpossible roundoff errors, and (b) it may also create other types ofnonlinear effects. Finally, using the inversion also requires additionalcomputations, which may be undesirable in certain cases.

Even without the inversion, in the regions with valleys, i.e., whereƒ(i,j)≈0, for both signal models, the scanner pattern can also berelatively easily estimated because then g(i,j)≈s(i,j). However, ƒ(i,j)may be as close to 0 as to make the approximation g(i,j)≈s(i,j)sufficiently accurate only for very small part of the pixels in an imagecontaining a fingerprint. In this case, therefore, the subsequent signalprocessing may use only that part of the image for which thisapproximation is sufficiently accurate or use larger part of the imagewhere the approximation is not. Either approach may degrade the overallperformance but is still possible and can be used in certain cases.

D.3 Scanner Authentication Modules for the 2D Wavelet Method

The two-dimensional (2D) wavelet method is suited for area scanners(also known as touch scanners). For these scanners, the fingertip isplaced on the scanner platen without the fingertip being moved, and thescanner acquires a static image (a “snapshot”) of the fingerprint suchthat each pixel of the digital image represents a miniscule area at aunique location of the fingerprint.

FIG. 8 shows a conceptual diagram of signal processing modules in whichthe signal g, the image, is processed to produce the signal d, thescanner verification decision, along with the interface signals amongthe modules.

D.3.1 Preprocessing Module

The Preprocessing Module is shown as block 401 in FIG. 8, and has asinput the signal g, which represents the pixel values g(i,j) of theimage, and as output the signal u, which is a two-dimensional signalwith the same size as g.

In one exemplary implementation, henceforth referred to as direct signalmode of the Preprocessing Module, the output signal is equal to itsinput signal:

u(i,j)=g(i,j).  (22)

In an alternative exemplary implementation, henceforth referred to asinverse signal mode of the Preprocessing Module, the output signal isthe inverse of the input signal:

$\begin{matrix}{{u\left( {i,j} \right)} = \left\{ {\begin{matrix}\frac{1}{g\left( {i,j} \right)} & {{{for}\mspace{14mu} {g\left( {i,j} \right)}} \neq 0} \\1 & {{{for}\mspace{14mu} {g\left( {i,j} \right)}} = 0}\end{matrix}.} \right.} & (23)\end{matrix}$

The preferred implementation uses the inverse mode of the PreprocessingModule.

D.3.2 Extraction Module

The Extraction Module is shown as block 402 in FIG. 8, and has as inputthe signal u, which represents the pixel values u(i,j) of the image, andas output the signal x, which is a two-dimensional signal of the samesize as u. However, the Extraction Module may process only part of theinput signal u, in which case the output the signal x will have adifferent size than the input signal u.

Wavelets are mathematical functions which are scaled and translatedcopies of a finite-length waveform. The representation of signals bywavelets is called wavelet transform and is used to analyze signals bysimultaneously revealing their frequency and location characteristics.The discrete wavelet transform decomposes a signal into frequencysubbands at different scales, and in this way, it allows the signalcharacteristics in each subband to be analyzed separately and modifiedon purpose so that the signal will possess certain desired propertiesafter it is reconstructed.

FIG. 19 shows an exemplary illustrative architecture of a single-leveltwo-dimensional discrete wavelet decomposition and reconstruction. Lo_Dand Hi_D are the low-pass and high-pass decomposition filters,respectively, and Lo_R and Hi_R are the low-pass and high-passreconstruction filters, respectively. The coefficients of these filtersare specifically designed such that as to perform the desired waveletprocessing of the signal. The input image is filtered sequentially byrows and by columns, and also downsampled, which results into computingthe wavelet coefficients in the corresponding subbands. After that, itis again filtered, and also upsampled, to reconstruct the output image.Today, this architecture can be implemented efficiently both in hardwareand in software.

The input signal u is decomposed by processing it with single-leveltwo-dimensional (2-D) discrete wavelets and then is reconstructed aftersetting to zero the wavelet coefficients of at least the LL (low-low)subband (i.e., the approximation coefficients); the wavelet coefficientsof the LH (low-high) subband, the HL (high-low) subband, and the HH(high-high) subband, and any combination of any two, but not all three,of these three subbands can also be set to zero. The reconstructed inthis way signal is henceforth referred to as residual and corresponds tosignal x in FIG. 8.

Using biorthogonal wavelets with decomposition order 5 andreconstruction order 1 provides excellent results, but other orders canalso be used. Using other types of wavelets such as Daubechies orsymlets also provides very good results when their order is 2 (i.e.,4-tap filter length); however, other orders of these wavelets can alsobe used. In addition, the present invention is not limited to thewavelets specified herein and other wavelets can also be used. Thepresent invention is also not limited to using single-level waveletdecomposition and reconstruction as using higher levels is alsopossible; however, a wavelet decomposition and reconstruction at ahigher level is typically more computationally expensive.

D.3.3 Postprocessing Module

The Postprocessing Module is shown as block 403 in FIG. 8, and has asinput the signal x and as output the signal p, which is atwo-dimensional signal with the same size as x.

In one exemplary implementation, henceforth referred to as direct signalmode of the Postprocessing Module, the output signal is equal to itsinput signal:

p(i,j)=x(i,j).  (24)

In an alternative exemplary implementation, henceforth referred to asinverse signal mode of the Postprocessing Module, the output signal isthe inverse of the input signal:

$\begin{matrix}{{p\left( {i,j} \right)} = \left\{ {\begin{matrix}\frac{1}{x\left( {i,j} \right)} & {{{for}\mspace{14mu} {x\left( {i,j} \right)}} \neq 0} \\1 & {{{for}\mspace{14mu} {x\left( {i,j} \right)}} = 0}\end{matrix}.} \right.} & (25)\end{matrix}$

D.3.4 Masking Module

The Masking Module is shown as block 404 in FIG. 8, and has as input thesignal p and as output the signal y, which is a two-dimensional signalof the same size as p.

The output signal y is constructed by taking those pixels from the inputsignal p that have magnitudes smaller than or equal to a predeterminedvalue θ and marking the rest of pixels as not useful:

$\begin{matrix}{{y\left( {i,j} \right)} = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} {{p\left( {i,j} \right)}}} \leq \theta} \\0 & {otherwise}\end{matrix} \right.} & (26)\end{matrix}$

When y(i,j)=1 in Equation (26), the corresponding pixel will be used infurther processing; otherwise, when y(i,j)=0 the pixel will not be used.The predetermined value θ is chosen as result of optimization for adesired False Accept Rate and False Reject Rate, and it depends on thetype and order of wavelets used in the Extraction Module. For SignalModel A, θ can be chosen from about 2 to about 6 (θ is not necessarilyinteger); for biorthogonal wavelets with decomposition order 5 andreconstruction order 1, choosing θ=4 provides a good overallperformance. For the inverse mode of the Preprocessing Module when usingthese wavelets, choosing θ from about 0.00005 to about 0.0003 ispossible, with good results achieved at about 0.0001.

D.3.5 Matching Module

The Matching Module is shown as block 406 in FIG. 8, and has as inputthe signal y, which represents the pixel values y(i,j), and as outputthe signal d.

Let x_(e) denote the output signal of the Extraction Module and y_(e)denote the output signal of the Masking Module when the input signal gis an image acquired during the scanner enrolment. Let x_(q) denote theoutput signal of the Extraction Module and y_(q) denote the outputsignal of the Masking Module when the input signal g is an imageacquired during the scanner verification. When the inverse mode of thePostprocessing Module is used, p_(e) and p_(q) are used instead of x_(e)and x_(q), respectively. Using the signals x_(e), y_(e), x_(q), andy_(q), the Matching Module: (i) selects the common pixel indices markedas useful in the signals y_(e) and y_(q), (ii) quantifies in a score thesimilarity between the two signals x_(e) and x_(q) for these commonpixel indices, and (iii) produces a decision via the output signal d asto whether the two images have been acquired with the same fingerprintscanner by comparing this score with a threshold. When the output signald takes the value 1, this indicates scanner match; when it takes value0, this indicates scanner non-match; and when it takes value (−1), thisindicates that a decision on matching/non-matching cannot be made and anew query image must be acquired. If multiple images are used for thescanner enrollment and/or the scanner verification, the methods forcomputing the score are as previously disclosed.

The two-dimensional signals x_(e), y_(e), x_(q), and y_(q), each havingI rows and J columns, are read column-wise, column by column, andtransformed into the one-dimensional signals x′_(e), y′_(e), x′_(q), andy′_(q), respectively, each having (I,J) elements. For example:

$\begin{matrix}\begin{matrix}{{{x_{e}^{\prime}(1)} = {x_{e}\left( {1,1} \right)}};{{x_{e}^{\prime}\left( {I + 1} \right)} = {x_{e}\left( {1,2} \right)}};} \\{{{x_{e}^{\prime}(2)} = {x_{e}\left( {2,1} \right)}};{{x_{e}^{\prime}\left( {I + 2} \right)} = {x_{e}\left( {2,2} \right)}};} \\\cdots \\{{{x_{e}^{\prime}(I)} = {x_{e}\left( {I,1} \right)}};{{x_{e}^{\prime}\left( {2 \cdot I} \right)} = {x_{e}\left( {I,2} \right)}};{{etc}.}}\end{matrix} & (27)\end{matrix}$

Reading the elements row-wise is also possible.

The selection of the common pixel indices marked as useful in thesignals y′_(e) and y′_(q) produces the signal y′_(m) defined as:

$\begin{matrix}{{y_{m}^{\prime}(k)} = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} {y_{e}^{\prime}(k)}} = {{1\mspace{14mu} {and}\mspace{14mu} {y_{q}^{\prime}(k)}} = 1}} \\0 & {otherwise}\end{matrix} \right.} & (28)\end{matrix}$

where the index k is an integer running from 1 to (I,J). Let D be theset of all indices k for which y′_(m)(k)=1, and let N_(D) be the numberof elements in this set D.

If N_(D) is less than about 100, the Matching Module produces (−1) asthe output signal d, which indicates that the number of common pixelindices is insufficient to compute a reliable similarity score and tomake a decision thereof. In this case, acquiring a new query image isnecessary.

Quantifying in a score the similarity between the two signals x′_(e) andx′_(q) for the common pixel indices as computed in the signal y′_(m) canbe done in many ways; three possible implementations are specifiedbelow. Recommending specific decision thresholds for each of them,however, is difficult. Conventionally, the decision threshold is theresult of optimization and tests with many images and depends on thedesired False Accept Rate and False Reject Rate. In the methodsdisclosed herein, the mean of the distribution of the scores for imagesacquired with the same scanner when different implementations of modulesare used and their parameters are set may vary in wide ranges: fromabout 0.15 to over 0.6, and although the mean of distribution of thescores for images acquired with the different scanners is typicallyclose to 0, providing general guidelines for the decision thresholds isdifficult. Therefore, we recommend determining the thresholds afterexperimentation and tests.

D.3.5.1 Normalized Correlation Implementation

First, the norms of the signals x′_(e) and x′_(q) for the indices in theset D are computed by:

$\begin{matrix}{{{x_{e}^{\prime}}} = \sqrt{\sum\limits_{k \in D}{{x_{e}^{\prime}(k)}}^{2}}} & (29) \\{{{x_{q}^{\prime}}} = {\sqrt{\sum\limits_{k \in D}{{x_{q}^{\prime}(k)}}^{2}}.}} & (30)\end{matrix}$

If any of the norms ∥x′_(e)∥ or ∥x′_(q)∥ are equal to zero, the MatchingModule produces 0 as an output signal d and does not perform furthercomputations. Otherwise, the similarity score z^((nc)) is computed by:

$\begin{matrix}{z^{({nc})} = {\frac{\sum\limits_{k \in D}\; {{x_{e}^{\prime}(k)}{x_{q}^{\prime}(k)}}}{{{x_{e}^{\prime}}} \cdot {{x_{q}^{\prime}}}}.}} & (31)\end{matrix}$

The output signal d is then computed by comparing the similarity scorez^((nc)) with a predetermined threshold:

$\begin{matrix}{d = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} z^{({nc})}} \geq \tau^{({nc})}} \\0 & {otherwise}\end{matrix} \right.} & (32)\end{matrix}$

The decision threshold τ^((nc)) is result of optimization and tests withmany images and depends on the desired False Accept Rate and FalseReject Rate.

D.3.5.2 Correlation Coefficient Implementation

First, the zero-mean signals {tilde over (x)}′_(e) and {tilde over(x)}′_(q) for the indices k in the set D are computed by:

$\begin{matrix}{{{\overset{\sim}{x}}_{e}^{\prime}(k)} = {{x_{e}^{\prime}(k)} - {\frac{1}{N_{D}}{\sum\limits_{k \in D}\; {x_{e}^{\prime}(k)}}}}} & (33) \\{{{\overset{\sim}{x}}_{q}^{\prime}(k)} = {{x_{q}^{\prime}(k)} - {\frac{1}{N_{D}}{\sum\limits_{k \in D}\; {x_{q}^{\prime}(k)}}}}} & (34)\end{matrix}$

where the index k runs through all elements in the set D. The values of{tilde over (x)}′_(e)(k) and {tilde over (x)}′_(q)(k) for indices k thatdo not belong to the set D can be set to 0 or any other number becausethey will not be used in the computations that follow.

Next, the norms of the signals {tilde over (x)}′_(e) and {tilde over(x)}′_(q) for the indices k in the set D are computed by:

$\begin{matrix}{{{\overset{\sim}{x}}_{e}^{\prime}} = \sqrt{\sum\limits_{k \in D}\; {{x_{e}^{\prime}(k)}}^{2}}} & (35) \\{{{\overset{\sim}{x}}_{q}^{\prime}} = {\sqrt{\sum\limits_{k \in D}\; {{x_{q}^{\prime}(k)}}^{2}}.}} & (36)\end{matrix}$

If any of the norms ∥{tilde over (x)}′_(e)∥ or ∥{tilde over (x)}′_(q)∥are equal to zero, the Matching Module produces 0 as an output signal dand does not perform further computations. Otherwise, the similarityscore z^((cc)) is computed by:

$\begin{matrix}{z^{({cc})} = \frac{\sum\limits_{k \in D}\; {{{\overset{\sim}{x}}_{e}^{\prime}(k)}{{\overset{\sim}{x}}_{q}^{\prime}(k)}}}{{{\overset{\sim}{x}}_{e}^{\prime}} \cdot {{\overset{\sim}{x}}_{q}^{\prime}}}} & (37)\end{matrix}$

The output signal d is then computed by comparing the similarity scorez^((cc)) with a predetermined threshold:

$\begin{matrix}{d = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} z^{({cc})}} \geq \tau^{({cc})}} \\0 & {otherwise}\end{matrix} \right.} & (38)\end{matrix}$

The decision threshold τ^((cc)) is result of optimization and tests withmany images and depends on the desired False Accept Rate and FalseReject Rate. It lies in the range from about 0.05 to about 0.30 forSignal Model A in direct mode of the Preprocessing and PostprocessingModules when the Extraction Module uses biorthogonal wavelets withdecomposition order 5 and reconstruction order 1 and θ=4. It also liesin the range from about 0.10 to about 0.50 for Signal Model A in inversemode of the Preprocessing and direct mode of the Postprocessing Modulewhen the Extraction Module uses biorthogonal wavelets with decompositionorder 5 and reconstruction order 1 and θ=0.0001.

The preferred implementation uses the Correlation CoefficientImplementation.

D.3.5.3 Relative Mean Square Error Implementation

First, the norm of the signal x′_(e) for the indices in the set D iscomputed as specified by:

$\begin{matrix}{{x_{e}^{\prime}} = {\sqrt{\sum\limits_{k \in D}\; {{x_{e}^{\prime}(k)}}^{2}}.}} & (39)\end{matrix}$

If the norm ∥x′_(e)∥ is equal to zero, the Matching Module produces 0 asan output signal d and does not perform further computations. Otherwise,the similarity score z^((rmse)) is computed by:

$\begin{matrix}{z^{({rmse})} = {\frac{\sqrt{\sum\limits_{k \in D}\; \left\lbrack {{x_{e}^{\prime}(k)} - {x_{q}^{\prime}(k)}} \right\rbrack^{2}}}{x_{e}^{\prime}}.}} & (40)\end{matrix}$

The output signal d is then computed by comparing the similarity scorez^((rmse)) with a predetermined threshold:

$\begin{matrix}{d = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} z^{({rmse})}} \leq \tau^{({rmse})}} \\0 & {otherwise}\end{matrix} \right.} & (41)\end{matrix}$

The decision threshold z^((rmse)) is result of optimization and testswith many images and depends on the desired False Accept Rate and FalseReject Rate.

D.3.6 Using Multiple Images

The exemplary implementations described in Section D.3 are capable ofusing a single image for the scanner enrolment and a single image forthe scanner verification and this is preferred because (a) it requiresthe least amount of computations and (b) it is the most secure as itdetermines if two images are taken with the same scanner or not withoutany additional images. However, variations are also possible. Forexample, it is typical for the biometric systems to capture three imagesand use them for enrolling the biometric information. Similarly, anotherexemplary implementation allows using multiple images for the scannerenrolment and/or multiple images for the scanner verification. This mayimprove the overall accuracy of the scanner authentication.

In general, the methods for processing multiple images are as previouslydisclosed. Herein, we disclose several exemplary illustrativeimplementations. Let the number of enrolled images be E and the outputsignals of the Extraction Module (or the Postprocessing Module, in whichcase the signal is p instead of x) and the Masking Module, when theenrolled image with index r is being processed, be x_(r) and y_(r),respectively. In the preferred exemplary implementation, the similarityscores for each pair consisting of one enrolled image and the queryimage are averaged and the resulting average similarity score is used toproduce a decision. Thus, if the similarity score between the queryimage and the enrolled image with index r is denoted by z_(r) which iscomputed using Equation (31), (37), or (40), then the average similarityscore z_(a) is:

$\begin{matrix}{z_{a} = {\frac{1}{E}{\sum\limits_{r = 1}^{E}\; {z_{r}.}}}} & (42)\end{matrix}$

Finally, the output signal d of the Matching Module is computed usingEquation (32), (37), or (41), depending on which implementation of theMatching Module is used for computing the similarity scores z_(r).

Another implementation computes an “average” enrolled scanner patternfrom all enrolled images and uses this “average” enrolled scannerpattern in the Matching Module. First, the “average” mask y′_(a) iscomputed by:

$\begin{matrix}{{y_{a}^{\prime}(k)} = {\prod\limits_{r = 1}^{E}\; {y_{r}^{\prime}(k)}}} & (43)\end{matrix}$

where k is an integer running from 1 to (I,J). Then the “average”scanner pattern is computed by:

$\begin{matrix}{{x_{a}^{\prime}(k)} = {\frac{1}{N_{a}}{\sum\limits_{r = 1}^{E}\; {{y_{a}^{\prime}(k)}{x_{r}^{\prime}(k)}}}}} & (44)\end{matrix}$

where N_(a) is the number of elements in y′_(a) for which y′_(a)(k)=1.Next, x′_(a) is used instead of the signal x′_(e) and y′_(a) is usedinstead of the signal y′_(e) in the Matching Module. The performance ofthis implementation may possibly be suboptimal in some cases because oftwo reasons: (1) since the signals y′_(r) for different indices r (andthus different enrolled images) may be considerably different from oneanother, the “average” mask y′_(a), which essentially is a logical ANDof all y′_(r), may have very few non-zero elements, which in turn mayresult in fewer than the sufficient number of pixels to be used in theMatching Module, and (2) the “average” signal x′_(a) may becomeconsiderably distorted for some pixels and this may result in falsescanner match or false scanner non-match decisions.

All implementations of the Matching Module can be used in combinationwith the implementations of the modules that precede it in theconceptual signal flow diagram shown in FIG. 8. However, differentcombinations of module implementations may provide different overallperformance.

D.3.7 Implementations and Performance

All implementations of the {Preprocessing Module, Extraction Module,Postprocessing Module, Masking Module, Matching Module} can be used incombination with any of the implementations of the modules that precedethis current module in the conceptual signal flow diagram depicted inFIG. 8. However, different combinations of module implementations mayprovide different overall performance.

One well-performing exemplary illustrative non-limiting combination ofmodule implementations is shown in FIG. 10. The flowchart 450 disclosesthe signal processing of this implementation using a single enrolledimage g_(e), acquired and processed during the scanner enrolment, and asingle query image g_(q), acquired and processed during the scannerverification. Although g_(e) and g_(q) are processed at different times,the consecutive processing steps are identical, and therefore hereinthey are discussed simultaneously. g_(e) and g_(q) are first processedby the Preprocessing Module 401, operating in its inverse signal mode.Its output signals u_(e) and u_(q) are then processed by the ExtractionModule 402 that uses biorthogonal wavelets with decomposition order 5and reconstruction order 1 with the LL subband coefficients set to zero.Its output signals x_(e) and x_(q) are processed by the PostprocessingModule 403, operating in its direct signal mode. Its output signalsp_(e) and p_(q) are processed by the Masking Module 404 using 0.0001 forthe predetermined value θ. Finally, the Matching Module 406 computes thecorrelation coefficient and produces the signal d, based on which adecision for scanner match or scanner nonmatch is made.

The disclosed herein signal processing modules can be implemented in theexemplary system 200 shown in FIG. 5 entirely by software programming ofthe processor 202, entirely in hardware, or some modules by softwareprogramming of the processor 202 and some modules in hardware. Forexample, the Extraction Module and the Masking Module can be implementedin the digital hardware 208, while the Preprocessing Module, thePostprocessing Module, and the Matching Module can be implemented insoftware that runs on one or more processors 202.

FIG. 16 shows the normalized histograms (integrating to 1) of thecorrelation coefficients and the scanner authentication decisions of theexemplary implementation shown in FIG. 10 when the query image has beenacquired with the authentic scanner and when the query image has beenacquired with an unauthentic scanner. The tested images have beenacquired with 22 area capacitive scanners of AuthenTec, taken from all10 fingers of 2 individuals, with 2 images per finger (880 images intotal). Only a single image was used for scanner enrolment and a singleimage for scanner verification and each image was matched against allother images. As the figure shows, the two histograms are clearlyseparated and far apart from each other. With the exemplary decisionthreshold of 0.30 (also shown in FIG. 16), no decision errors are made.

The performance (as accuracy) of the exemplary implementations we justdescribed earlier is not the best one possible the methods disclosedherein can deliver; rather, it is just an example for their potential.The methods should be considered as a set of tools for achieving thepurpose of scanner authentication. Therefore, the modules and theirmodes as to be implemented in a particular target application should bechosen and their parameters optimized once the specific applicationrequirements and constraints are defined.

The wavelet method disclosed herein may provide higher accuracy incertain applications and environments. It is also universal as it doesnot assume any specific signal model and its parameters, and thereforeit may work with other types of fingerprints scanners and even otherimage acquisition devices. Furthermore, although developed for areascanners, it may also prove to be suited for certain swipe scanners. Inaddition, the wavelet processing may come with little additional cost asmany fingerprint systems currently used already implement wavelettransforms. Its simplicity and computational efficiency among methods ofsimilar type also make it attractive. A possible downside is that,although it operates on images containing fingerprints, it may rely onthe areas of the image that are not covered by the fingerprint pattern,possibly lightening its security. Its robustness under differentenvironmental conditions may also need to be further studied.

As the methods do not require changes in the fingerprint scanners, theycan be implemented in systems that have already been manufactured andeven sold to customers by software and/or programmable hardwareupgrades. Furthermore the implementations of the methods disclosedherein do not incur additional material and manufacturing costs.

D.4 Scanner Authentication Modules for the Averaging Method

The averaging method is suited for swipe scanners (also known as slideor sweep scanners). In these scanners, a line, being a row or a column,of sensor elements performs an instant scan of a tiny area of thefingertip skin and converts the readings into a line (a vector) ofpixels. As the fingertip is swiped over this line of sensor elements, asequence of such lines of pixels is produced, which sequence is thenassembled (and possibly further enhanced) to construct a two-dimensionalfingerprint image.

The signal processing modules for this method have been designedassuming that the signals follow Signal Model A and tested with theswipe capacitive fingerprint scanners of AuthenTec. In comparison withthe area fingerprint scanners, the swipe fingerprint scanners have threemajor differences: (a) they have much fewer number of sensing elements(in the order of 100 to 200, typically 144), in contrast with the areascanners which typically have tens of thousands, (b) acquiring imageswith a predetermined object (e.g., air) cannot be done for all pixelelements at once, and (c) the images constructed by combining thescanned lines sometimes may contain artifacts from this construction andpossibly also image enhancements.

The swipe scanners we used, however, have two favorable properties: (a)the pixels in the image never saturate (“clip”) and (b) in a singleimage, each sensing element produces many (e.g., hundreds) pixel values,not only one pixel value as the area scanners do, and in this way, thescanner pattern of each sensing element gets “incorporated” in manypixels of the image, thus facilitating the process of estimating it. Theswipe scanners we used scan rows of pixels, and, therefore, by averagingalong the columns of pixels, the scanner pattern for each sensing pixelbecomes “stronger” and easier to estimate; this is the basis of theaveraging method.

FIG. 9 shows a conceptual diagram of signal processing modules in whichthe signal g, the image, is processed to produce the signal d, thescanner verification decision, along with the main interface signalsbetween the modules.

D.4.1 Preprocessing Module

The Preprocessing Module is shown as block 502 in FIG. 9, and has asinput the signal g, which represents the pixel values g(i,j) of theimage, and as output the signal u, which is a two-dimensional signalwith the same size as g.

In one exemplary implementation, henceforth referred to as direct signalmode of the Preprocessing Module, the output signal is equal to itsinput signal:

u(i,j)=g(i,j).  (45)

In an alternative exemplary implementation, henceforth referred to asinverse signal mode of the Preprocessing Module, the output signal isthe inverse of the input signal:

$\begin{matrix}{{u\left( {i,j} \right)} = \left\{ \begin{matrix}\frac{1}{g\left( {i,j} \right)} & {{{for}\mspace{14mu} {g\left( {i,j} \right)}} \neq 0} \\1 & {{{for}\mspace{14mu} {g\left( {i,j} \right)}} = 0}\end{matrix} \right.} & (46)\end{matrix}$

D.4.2 Averaging Module

The Averaging Module is shown as block 504 in FIG. 9. It computes theaverage values of the pixels along columns (or along rows, depending onthe scanning direction of the line of sensor elements) from its inputsignal u and produces these average values as its output signal p to beprocessed further in the subsequent modules. Thus, the input signal u istwo dimensional, whereas the output signal p is one dimensional, i.e., avector (a line) of values.

Let I be the total number of rows and J be the total number of columnsin g. Typically, the line of sensor elements in most swipe scanners isperpendicular to the length of the finger, and therefore the finger isswept over the scanner in the direction of finger's length. In thiscase, the sequentially produced lines of pixels form rows in thetwo-dimensional image g. Thus, the pixels in each column of g areproduced by one and the same sensing element, i.e., for a given (andfixed) column j and for all row indices i from 1 through I, the pixelsg(i,j) are produced by the sensing element with index j in the line of Jsensing elements. Alternatively, in scanners where the orientation ofthe line of sensor elements is along the length of the finger, for agiven (and fixed) row i and for all column indices j from 1 through J,the pixels g(i,j) are produced by the sensing element with index i inthe line of I sensing elements.

The Averaging Module computes the average of the pixel values producedby one and the same sensor element (in direct or in inverse, dependingon the mode of the Preprocessing module). Thus, for scanners in whichthe line of sensor elements is perpendicular to the length of thefinger, the averaging is along columns and the output signal p is:

$\begin{matrix}{{p(j)} = {\frac{1}{I}{\sum\limits_{i = 1}^{I}\; {u\left( {i,j} \right)}}}} & (47)\end{matrix}$

where j is from 1 to J. For scanners in which the orientation of theline of sensor elements is along the length of the finger, the averagingis along rows and the output signal p is:

$\begin{matrix}{{p(i)} = {\frac{1}{J}{\sum\limits_{j = 1}^{J}\; {u\left( {i,j} \right)}}}} & (48)\end{matrix}$

where i is from 1 to I. For clarity, in the discussion that follows, wedescribe the case when the line of sensor elements is perpendicular tothe length of the finger and thus the output signal is p(j) where j isfrom 1 to J. The alternative case is analogous.

The first key observation behind the averaging is that, in both directand inverse modes of the Preprocessing Module, the output signal p canbe approximated as comprising two additive components: (1) the first onerepresenting the scanner pattern s(j) at each column j and (2) thesecond one representing the average fingerprint row ƒ_(avg)(j), i.e.,the average of the fingerprint pattern along each column j. In inversesignal mode, this is straightforward; for example, by averaging h(i,j)of Equation (16) along columns, we obtain the average row h_(avg)(j):

$\begin{matrix}{{h_{avg}(j)} \approx {\frac{1}{s(j)} + {f_{avg}(j)}}} & (49)\end{matrix}$

The second key observation behind the averaging is that ƒ_(avg)(j) is aslowly varying function of the column index j due to the high spatialscanning resolution of scanners in comparison with the rate of change ofthe fingerprint pattern ƒ(i,j) (i.e., the sequence of valleys andridges) along rows (and also along columns). Consequently, the pixelvalues g(i,j) (as well as their inverses h(i,j)) at adjacent columnscannot differ by much, and more importantly, the averages along adjacentcolumns are close in value to each other.

Next, as already disclosed, the scanner pattern along rows (i.e., s(j)in this case) has approximately a Gaussian distribution, andconsequently, its inverse 1/s(j) has also approximately a Gaussiandistribution for the range of values in our case (see Equation (21)).Since ƒ_(avg)(j) is slowly varying in function of the column index j,1/s(j) is approximately Gaussian in function of the column index j, andthe two are independent, separating them can be done with simple signalprocessing, as disclosed in the Filtering Module herein.

This averaging works similarly also in the direct mode of thePreprocessing Module, but the analysis is much more involved. Byaveraging along columns, the average row g_(avg)(j) can also beapproximated as comprising two additive components, the first one beinga function of the scanner pattern s(j) and the second one—of the averagefingerprint row ƒ_(avg)(j):

g _(avg)(j)≈A[s(j)]+B└ƒ _(avg)(j)┘  (50)

where the functions A[•] and B[•] are approximately A[s(j)]≈const₁·s(j)and B└ƒ_(avg)(j)┘≈const₂·(ƒ_(avg)(j)−const₃), for some constants const₁,const₂, and const₃. The second component, B└ƒ_(avg)(j)┘, is believed tobe very small or at least slowly varying with the index j, whichcomponent a filter can remove and produce the first component A[s(j)],which in turn represents the scanner pattern. It is important to note,however, that the approximation in Equation (50) and the approximationsfor A[s(j)] and B[f_(avg)(j)] are sufficiently accurate not in general,but only: (a) for the characteristics and value ranges of the scannerpattern and of the fingerprint pattern ƒ(j) (which in this case variesfrom 0 to about 0.005) in the swipe capacitive scanners that we used and(b) in the context of the operation of the Filtering Module disclosedherein.

It is not necessary to average all rows (and respectively, columns) inan image, but only a few of them, which can save computational time. Theminimum number of rows used in the averaging needs to be determinedexperimentally and depending on the available computational power andtime, but for achieving a good performance, we recommend that at least100 rows (and respectively, columns) be averaged.

Some swipe scanners may employ more than one line of pixels, in whichcase the process of constructing a fingerprint image from the sequenceof lines of pixels may involve sophisticated signal processing, which istypically manufacturer proprietary.

D.4.3 Postprocessing Module

The Postprocessing Module is shown as block 505 in FIG. 9, and has asinput the signal p, which is the output signal of the Averaging Module,and as output the signal ν, which is a one-dimensional signal with thesame size as p.

In one exemplary implementation, henceforth referred to as direct signalmode of the Postprocessing Module, the output signal is equal to itsinput signal:

ν(j)=p(j).  (51)

In an alternative exemplary implementation, henceforth referred to asinverse signal mode of the Postprocessing Module, the output signal isthe inverse of the input signal:

$\begin{matrix}{{v(j)} = \left\{ {\begin{matrix}\frac{1}{p(j)} & {{{for}\mspace{14mu} {p(j)}} \neq 0} \\1 & {{{for}\mspace{14mu} {p(j)}} = 0}\end{matrix}.} \right.} & (52)\end{matrix}$

D.4.4 Filtering Module

The Filtering Module is shown as block 506 in FIG. 9. The FilteringModule filters the input signal ν to produce the output signal x, whichsignal x contains the scanner pattern. Due to the operation of theAveraging Module and also to the inverse mode of the PreprocessingModule when used, this filtering is performed using simple signalprocessing and essentially comprises two operations: (1) a smoothingoperation F(•) that smooths the input signal ν and (2) a subtractionoperation that subtracts this smoothed signal from the input signal ν,producing the output signal x:

x=v−F(v).  (53)

In this way, the smoothing also removes the (variable) mean of thescanner pattern and yields only the variable part of it. This variablepart can then be used in the Matching Module by comparing it to asimilarly produced signal that is derived from another image or as asource of randomness that is unique to the particular scanner, acquiredthe former image.

Let N denote the number of elements of the input signal ν. For scannersin which the line of sensor elements is perpendicular to the length ofthe finger, N=J, whereas for scanners in which the orientation of theline of sensor elements is along the length of the finger, N=I.

D.4.4.1 Padding and Windowing

Because of the finite length N of the input signal ν, the signalprocessing of the discontinuity at the beginning and at the end of ν maylead to unwanted artifacts, and, therefore, it is preferable thattechniques are employed to avoid these artifacts. Three such techniquesare included in this disclosure, henceforth referred to as computationshortening, replica padding, and constant padding, although using othertechniques is also possible. The replica padding method and constantpadding method are specified next. The computation shortening method isspecific for the particular implementation of the Filtering Module andspecified in the appropriate sections (see sections D.4.4.2 andD.4.4.3).

i. Replica Padding

-   -   The vector ν is extended to include zero and negative indices        and indices larger than N such that the added elements are        symmetric about the first and the last indices of ν:

$\begin{matrix}{{v(j)} = {{v\left( {2 - j} \right)}\mspace{14mu} {for}\mspace{14mu} j\mspace{14mu} {from}\mspace{14mu} \left( {- \left\lfloor \frac{M}{2} \right\rfloor} \right)\mspace{14mu} {to}\mspace{14mu} 0}} & (54)\end{matrix}$

$\begin{matrix}{{v(j)} = {{v\left( {{2\; N} - j} \right)}\mspace{14mu} {for}\mspace{14mu} j\mspace{14mu} {from}\mspace{14mu} \left( {N + 1} \right)\mspace{14mu} {to}\mspace{14mu} \left( {N + \left\lfloor \frac{M - 1}{2} \right\rfloor} \right)}} & (55)\end{matrix}$

-   -   The added elements in this extension can also be copies of the        first and last elements, respectively, of the vector ν in the        same order as they appear in ν.

ii. Constant Padding

-   -   The vector ν is extended to include zero and negative indices        and indices larger than N such that the added elements are set        to constants. The constants can be the first and last elements,        respectively, as follows:

$\begin{matrix}{{v(j)} = {{v(1)}\mspace{14mu} {for}\mspace{14mu} j\mspace{14mu} {from}\mspace{14mu} \left( {- \left\lfloor \frac{M}{2} \right\rfloor} \right)\mspace{14mu} {to}\mspace{14mu} 0}} & (56) \\{{v(j)} = {{v(N)}\mspace{14mu} {for}\mspace{14mu} j\mspace{14mu} {from}\mspace{14mu} \left( {N + 1} \right)\mspace{14mu} {to}\mspace{14mu} {\left( {N + \left\lfloor \frac{M - 1}{2} \right\rfloor} \right).}}} & (57)\end{matrix}$

-   -   The constants can also be other numbers in the grayscale level        range (from 0 to 255), but we recommend that the constants are        chosen to be close in value to the values of the first and the        last elements, respectively, as choosing constants significantly        different from them may lead to considerably degraded overall        performance.

Incorporating such techniques to avoid edge effect artifacts may seemunjustified, but actually it is important because the length N of signalν is relatively small (in the order of one to several hundreds) and suchartifacts may affect the estimate of the scanner pattern of about 10pixels, which is not negligible and may decrease the performance.Furthermore, because applying a fingertip tightly in the regions aroundthe boundaries of the scanner platen area (and in the two ends of theline of sensor elements in this respect) is difficult, the image pixelsin these regions typically contain no fingerprint pattern. Hence, theestimate of the scanner pattern in these regions can be made veryaccurate if such unwanted artifacts are avoided as disclosed herein.

Another important aspect of the processing in this module that mayincrease the overall performance is using a windowing function appliedto the signal being processed. By multiplying the pixel values by awindowing function (for example, see w(j) in Equation (59)), the pixelsclose to the current index of the signal being processed have higherweight in the computation. This is a technique for controlling the levelof the smoothing effect by placing larger weight on the pixels aroundthe center pixel than on the distant pixels and thus reducing the effectof the latter.

The windowing function w(j) of size M, for j being an integer from

$- \left\lfloor \frac{M}{2} \right\rfloor$

to can be:

$\left\lfloor \frac{M - 1}{2} \right\rfloor,$

-   -   i. A rectangular window (also known as Dirichlet window): w(j)=1    -   ii. A triangular window (also known as Bartlett window):

${w(j)} = {2\left( {1 - {\frac{2}{M}{j}}} \right)}$

-   -   iii. A Hann window (also known as Hanning or raised-cosine        window):

${w(j)} = {1 + {\cos \left( \frac{2\; j\; \pi}{M} \right)}}$

-   -   iv. A Hamming window:

${w(j)} = {2\left( {0.54 + {0.46 \cdot {\cos \left( \frac{2\; j\; \pi}{M} \right)}}} \right)}$

-   -   v. A Gaussian window:

${{w(j)} = {2 \cdot {\exp\left( {{- \frac{1}{2}}\left( \frac{j}{\frac{w_{0}\left( {M - 1} \right)}{2}} \right)^{2}} \right)}}},$

where w₀ is a suitably chosen value below 0.5.

Using other windowing functions is also possible. The windowing functionof choice has to satisfy or be normalized as to satisfy thenormalization condition:

$\begin{matrix}{{\sum\limits_{\underset{{w{(j)}}\mspace{14mu} {is}\mspace{14mu} {used}}{{for}\mspace{14mu} {all}\mspace{14mu} j\mspace{14mu} {for}\mspace{14mu} {which}}}\; {w(j)}} = M} & (58)\end{matrix}$

D.4.4.2 Low-Pass Filter Implementation of the Filtering Module

The smoothing operation in this implementation is performed by alow-pass filter whose cutoff frequency, order, and attenuation in thedifferent frequency bands are optimized for best performance. Thislow-pass filter includes, but is not limited to, Butterworth, Chebyshev,elliptic, and Bessel filters and filters having finite impulse response(FIR) and filters having infinite impulse response (IIR).

The low-pass filter of choice and disclosed herein is the windowedmoving-average filter because of its extreme implementation simplicityand the corresponding excellent overall performance. Generally, for apixel with index k sufficiently far from the beginning and end of theinput signal ν (which is a vector), i.e., such that the index (k+j) doesnot address elements outside the vector ν, the local mean ν^((lm)) iscomputed by:

$\begin{matrix}{{v^{({lm})}(k)} = {\frac{1}{M}{\sum\limits_{j = {- {\lfloor\frac{M}{2}\rfloor}}}^{\lfloor\frac{M - 1}{2}\rfloor}\; {{w(j)} \cdot {v\left( {k + j} \right)}}}}} & (59)\end{matrix}$

where M is a positive integer and determines the size of themoving-average window, w is a windowing function, and └.┘ is the floorfunction. Preferably, M is selected to be odd so that the window becomessymmetric about the index k, but selecting M to be even is alsopossible. Selecting M about 3 gives optimal results, but good overallperformance is also achieved for M in the range from 2 to about 7. Oncethe windowing function is selected, the size M of the moving-averagewindow may need to be adjusted for achieving optimal overallperformance.

For the pixels that are close to the beginning or the end of the vectorν, three techniques for computing the local mean ν^((lm)) are includedin the present disclosure; using other techniques, however, is alsopossible:

i. Computation Shortening

-   -   The sum in Equation (59) and the denominator in the coefficient        in front of it are adjusted so that only elements of the vector        ν are used in the computation. Thus, for the index k where

${k \leq {\left\lfloor \frac{M}{2} \right\rfloor \mspace{14mu} {or}\mspace{14mu} k} \geq \left( {N - \left\lfloor \frac{M - 1}{2} \right\rfloor + 1} \right)},$

the local mean vector ν^((lm)) is computed by:

$\begin{matrix}{{{v^{({lm})}(k)} = {\frac{1}{\left( {j_{\max} - j_{\min} + 1} \right)}{\sum\limits_{j = j_{\min}}^{j_{\max}}\; {{{w_{k}(j)} \cdot {v\left( {k + j} \right)}}\mspace{14mu} {where}}}}}{j_{\min} = {{{\max \left( {{- \left\lfloor \frac{M}{2} \right\rfloor},{1 - k}} \right)}\mspace{14mu} {and}\mspace{14mu} j_{\max}} = {{\min \left( {\left\lfloor \frac{M - 1}{2} \right\rfloor,{N - k}} \right)}.}}}} & (60)\end{matrix}$

In this case, the windowing function depends on the index k because thewindow is truncated and needs to be renormalized so that the sum of itselements equals (j_(max)−j+1). Therefore, for all values of j fromj_(min) to j_(max) as defined by Equation (60):

$\begin{matrix}{{{w_{k}(j)} = \frac{\left( {j_{\max} - j_{\min} + 1} \right) \cdot {w(j)}}{M}},} & (61)\end{matrix}$

where w is the chosen normalized windowing function. The computationshortening may lead to a slight degradation in the accuracy of the localmean estimate for the pixels where it is applied, but the distortion thecomputation shortening introduces is the smallest in comparison with theother techniques.

ii. Replica Padding

The vector ν is extended to include zero and negative indices andindices larger than N as specified in Equations (54) and (55). The addedelements in this extension can also be copies of the first and lastelements, respectively, of the vector ν in the same order as they appearin ν. Then the local mean vector ν^((lm)) is computed using Equation(59).

iii. Constant Padding

The vector ν is extended to include zero and negative indices andindices larger than N as specified in Equations (56) and (57). Then thelocal mean vector ν^((lm)) is computed using Equation (59).

Finally, the output signal x of the Filtering Module in thisimplementation is the difference between the input signal ν and thelocal mean signal ν^((lm)):

x(k)=ν(k)−ν^((lm))(k)  (62)

where k is the current pixel index, an integer from 1 to N.

FIG. 17 shows the input signal ν and the output signal x of theexemplary implementation shown in FIG. 11, in which the Filtering Moduleuses a moving-average filter and both the Preprocessing Module and thePostprocessing Module operate in their direct signal modes.

D.4.4.3 Adaptive Wiener Filter Implementation of the Filtering Module

Herewith we incorporate a summary of the theory of Wiener filters asdeveloped in Jae Lim, “Two-dimensional Image and Signal processing” forthe one-dimensional case. Let a signal p(k) and an additive noise q(k),where k is an integer, are two zero-mean second-order stationarydiscrete-time random processes, linearly independent of each other, andthe noisy observation r(k) is: r(k)=p(k)+q(k). The objective is findingthat linear time-invariant (or space-invariant) filter with a possiblyinfinite and possibly non-causal impulse response b(k) such that thelinear estimate {circumflex over (p)}(k) given the observation r(k),i.e., {circumflex over (p)}(k)=r(k)*b(k), is closest to the signal p(k)in mean-square error sense: E[|p(k)−{circumflex over (p)}(k)|²]. Thediscrete-time Fourier transform of the linear time-invariant filter b(k)that minimizes the mean square error is:

${{B(\omega)} = \frac{S_{p}(\omega)}{{S_{p}(\omega)} + {S_{q}(\omega)}}},$

where S_(p)(ω) and S_(q)(ω) are the power spectral densities of thesignal p(k) and the noise q(k), respectively, and ω is the angularfrequency. If p(k) and q(k) are Gaussian random processes, then theWiener filter is also the optimal nonlinear mean-square error estimator.

In essence, the Wiener filter preserves the high SNR frequencycomponents and suppresses the low SNR frequency components. If we define

${\rho (\omega)}\overset{\Delta}{=}\frac{S_{p}(\omega)}{S_{q}(\omega)}$

to be the signal-to-noise ratio (SNR) in function of the frequency, thenthe Wiener filter transfer function is:

${B(\omega)} = {\frac{\rho (\omega)}{{\rho (\omega)} + 1}.}$

At the frequencies where the signal is much stronger than the noise,i.e., where ρ(ω)>>1, the transfer function is B(ω)≈1, and theobservation r(k) passes through the filter almost unchanged. On theother hand, the Wiener filter almost completely suppresses, i.e.,B(ω)≈0, the frequency components at which the signal is much weaker thanthe noise, i.e., where ρ(ω)≈0. If the signal p(k) has a nonzero meanμ_(p) and the noise q(k) has a nonzero mean μ_(q), they have to besubtracted from the observation r(k) before filtering it.

When the impulse response of the Wiener filter changes depending on thelocal characteristics of the signal that is being processed, the filterbecomes time variant (or space variant). Thus, instead of using constant(for all indices k) power spectral densities for the signal and thenoise, they can be estimated locally; furthermore, the means of thesignal and the noise can be estimated locally as well. Depending on howthese quantities are estimated, many variations are possible, but thesimplest one is when the local power spectral densities of both thesignal and the noise are constant in function of the frequency, i.e.,the signal and the noise are “white.” When the signal and the noise arezero mean, their power spectral densities are equal to their (local)variances:

S _(p)(ω)=σ_(p) ² and S _(q)(ω)=σ_(q) ²

where σ_(p) ² and σ_(q) ² are the variances of the signal and the noise,respectively. In this case, the frequency response of the Wiener filteris constant in function of the frequency, and thus its impulse responseis a scaled Dirac delta function:

${b(k)} = {\frac{\sigma_{p}^{2}}{\sigma_{p}^{2} + \sigma_{q}^{2}}{\delta (k)}}$

where δ(k) is the Dirac delta function. Moreover, the filtering alsodepends on the relative relationship between the local variance of thesignal and the noise: where the signal local variance σ_(P) ² is smallerthan the noise local variance σ_(q) ², the filter suppresses the noiseand thus the filter output is approximately equal to the local mean ofthe signal. On the other hand, where the signal local variance σ_(p) ²is larger than the noise local variance σ_(q) ², the filter leaves theinput signal almost unchanged. Since the signal (local) variance is notknown and generally is difficult to be estimated, in practice anestimate for the variance of the noisy observation r(k) is used insteadbecause σ_(r) ²=σ_(p) ²+σ_(q) ². Putting all things together yields thefollowing expression for the estimate {circumflex over (p)}(k) of thesignal p(k);

${\hat{p}(k)} = {{\mu_{p}(k)} + {\frac{\max \left( {0,{{\sigma_{r}^{2}(k)} - \sigma_{q}^{2}}} \right)}{\max \left( {{\sigma_{r}^{2}(k)},\sigma_{q}^{2}} \right)}\left( {{p(k)} - {\mu_{p}(k)}} \right)}}$

where σ_(r) ²(k) is the local variance of the observation r(k), andμ_(p)(k) is the local mean of the signal p(k), which is also equal tothe local mean μ_(r)(k) of the observation r(k) since the noise q(k) iszero mean. Assumed to be known is only the variance σ_(q) ² of thenoise; σ_(r) ²(k) and μ_(r)(k) (and thus also p_(p) (k)) are estimatedfrom the observation r(k). The output of the adaptive Wiener filter isthe estimate μ_(p)(k), which is a smoothed version of the signal p(k).

The input signal (and vector) ν is processed in the following fivesteps:

D.4.4.3.1 Computing the Local Mean

Generally, for a pixel with index k sufficiently far from the beginningand the end of ν, i.e., such that the index (k+j) does not addresselements outside ν, the local mean vector ν^((lm)) is computed by:

$\begin{matrix}{{v^{({lm})}(k)} = {\frac{1}{M}{\sum\limits_{j = {- {\lfloor\frac{M}{2}\rfloor}}}^{\lfloor\frac{M - 1}{2}\rfloor}\; {{w(j)} \cdot {v\left( {k + j} \right)}}}}} & (63)\end{matrix}$

where M is a positive integer and determines the size of themoving-average window, w is a windowing function, and └.┘ is the floorfunction. Preferably, M is selected to be odd so that the window becomessymmetric about the index k, but selecting M to be even is alsopossible. Selecting M to be about 3 gives optimal results, but goodoverall performance is also achieved for M in the range from 2 to about7. Once the windowing function is selected, the size M of themoving-average window may need to be adjusted for achieving optimaloverall performance.

For the pixels that are close to the beginning or the end of the vectorν, three techniques for computing the local mean ν^((lm)) are includedin the exemplary implementations, although using other techniques isalso possible:

i. Computation-Shortening

-   -   The sum in Equation (63) and the denominator in the coefficient        in front of it are adjusted so that only elements of the vector        ν are used in the computation. Thus, for the index k where

${k \leq {\left\lfloor \frac{M}{2} \right\rfloor \mspace{14mu} {or}\mspace{14mu} k} \geq \left( {N - \left\lfloor \frac{M - 1}{2} \right\rfloor + 1} \right)},$

-   -   the local mean vector ν^((lm)) is computed by:

$\begin{matrix}{{{v^{({lm})}(k)} = {\frac{1}{\left( {j_{\max} - j_{\min} + 1} \right)}{\sum\limits_{j = j_{\min}}^{j_{\max}}\; {{w_{k}(j)} \cdot {v\left( {k + j} \right)}}}}}{where}{j_{\min} = {{{\max \left( {{- \left\lfloor \frac{M}{2} \right\rfloor},{1 - k}} \right)}\mspace{14mu} {and}\mspace{14mu} j_{\max}} = \; {\min \left( {\left\lfloor \frac{M - 1}{2} \right\rfloor,{N - k}} \right)}}}} & (64)\end{matrix}$

-   -   In this case, the windowing function depends on the index k        because the window is truncated and needs to be renormalized so        that the sum of its elements equals (j_(max)−j_(min)+1).        Therefore, for all values of j from j_(min) to j_(max) as        defined by Equation (64):

$\begin{matrix}{{{w_{k}(j)} = \frac{\left( {j_{\max} - j_{\min} + 1} \right) \cdot {w(j)}}{M}},} & (65)\end{matrix}$

-   -   where w is the chosen normalized windowing function. The        computation shortening may lead to slight degradation in the        accuracy of the local mean estimate for the pixels where it is        applied, but the distortion the computation shortening        introduces is the smallest in comparison with the other        techniques.

ii. Replica Padding

-   -   The vector ν is extended to include zero and negative indices        and indices larger than N as specified in Equations (54) and        (55). The added elements in this extension can also be copies of        the first and last elements, respectively, of the vector ν in        the same order as they appear in ν. Then the local mean vector        ν^((lm)) is computed using Equation (63).

iii. Constant Padding

-   -   Each vector ν is extended to include zero and negative indices        and indices larger than N as specified in Equations (56) and        (57). Then the local mean vector ν^((lm)) is computed using        Equation (63).

D.4.4.3.2 Computing the Local Square

Generally, for a pixel with index k sufficiently far from the beginningand the end of ν, i.e., such that the index (k+j) does not addresselements outside ν, the local square vector ν^((ls)) is computed by:

$\begin{matrix}{{v^{({ls})}(k)} = {\frac{1}{M}{\sum\limits_{j = {- {\lfloor\frac{M}{2}\rfloor}}}^{\lfloor\frac{M - 1}{2}\rfloor}\; {{w(j)} \cdot {v^{2}\left( {k + j} \right)}}}}} & (66)\end{matrix}$

where M is a positive integer and determines the size of the window, wis a windowing function, and └.┘ is the floor function. Preferably, M isselected to be odd so that the window becomes symmetric about the indexk, but selecting M to be even is also possible. Selecting M to be about3 gives optimal results, but good overall performance is also achievedfor M in the range from 2 to about 7. Once the windowing function isselected, the size M of the window may need to be adjusted for achievingoptimal overall performance.

For the pixels that are close to the beginning or the end of ν, threetechniques for computing the local square vector ν^((ls)) are includedin the exemplary implementations, although using other techniques isalso possible:

i. Computation Shortening

-   -   The sum in Equation (66) and the denominator in the coefficient        in front of it are adjusted so that only elements of the vector        ν are used in the computation. Thus, for the index k where

${k \leq {\left\lfloor \frac{M}{2} \right\rfloor \mspace{14mu} {or}\mspace{14mu} k} \geq \left( {N - \left\lfloor \frac{M - 1}{2} \right\rfloor + 1} \right)},$

-   -   the local square vector ν^((ls)) is computed by:

$\begin{matrix}{{{v^{({ls})}(k)} = {\frac{1}{\left( {j_{\max} - j_{\min} + 1} \right)}{\sum\limits_{j = j_{\min}}^{j_{\max}}\; {{w_{k}(j)} \cdot {v^{2}\left( {k + j} \right)}}}}}{where}{j_{\min} = {{{\max \left( {{- \left\lfloor \frac{M}{2} \right\rfloor},{1 - k}} \right)}\mspace{14mu} {and}\mspace{14mu} j_{\max}} = \; {{\min \left( {\left\lfloor \frac{M - 1}{2} \right\rfloor,{N - k}} \right)}.}}}} & (67)\end{matrix}$

-   -   In this case, the windowing function depends on the index k        because the window is truncated and needs to be renormalized so        that the sum of its elements equals (j_(max)−j_(min)+1).        Therefore, for all values of j from J_(min) to j_(max) as        defined by Equation (67):

$\begin{matrix}{{{w_{k}(j)} = \frac{\left( {j_{\max} - j_{\min} + 1} \right) \cdot {w(j)}}{M}},} & (68)\end{matrix}$

-   -   where w is the chosen windowing function. The computation        shortening may lead to slight degradation in the accuracy of the        local square estimate for the pixels where it is applied, but        the distortion the computation shortening introduces is the        smallest in comparison with the other techniques.

ii. Replica Padding

The vector ν is extended to include zero and negative indices andindices larger than N as specified in Equations (54) and (55). The addedelements in this extension can also be copies of the first and lastelements, respectively, of the vector ν in the same order as they appearin ν. Then the local square ν^((ls)) is computed using Equation (66).

iii. Constant Padding

Each vector ν is extended to include zero and negative indices andindices larger than N as specified in Equations (56) and (57). Then thelocal square ν^((ls)) is computed using Equation (66).

D.4.4.3.3 Computing the Local Variance Vector

For each pixel with index k, where k is from 1 to N, each element of thelocal variance vector ν^((lv)) is computed by:

ν^((lv))(k)=ν^((ls))(k)−(ν^((lm))(k))²  (69)

D.4.4.3.4 Computing the Scaling Coefficient Vector

For each pixel with index k, where k is from 1 to N, each element of thescaling coefficient vector d is computed by:

$\begin{matrix}{{d(k)} = \left( \frac{\max \left( {0,{{v^{({lv})}(k)} - \sigma_{w}^{2}}} \right)}{\max \left( {{v^{({lv})}(k)},\sigma_{w}^{2}} \right)} \right)^{\beta_{w}}} & (70)\end{matrix}$

where σ_(w) ² is the Wiener variance and β_(w) is the Wiener betacoefficient. Since in Equation (70), the numerator is always smallerthan the denominator, by raising the ratio to power β_(w), chosen to begreater than 1, the scaling coefficient d(k) will be smaller than whenβ_(w) is 1. Conversely, by raising the ratio to power β_(w) chosen to besmaller than 1, the scaling coefficient d(k) will be greater then whenβ_(w) is 1. Thus, the Wiener filter beta coefficient β_(w) controls therelative weight put on the scaling factor with respect to the differencebetween the local variance ν^((lv))(k) and the Wiener filter varianceσ_(w) ². A Wiener filter beta coefficient β_(w) of 1 provides goodoverall performance along with simple implementation since no raising topower is computed in this case, but other values of β_(w) can also beused; β_(w) is not necessarily integer.

The Wiener filter variance σ_(w) ² is a critically important parameterthat determines the overall performance. The best value for σ_(w) ² istypically result of optimization and tests (with multiple scanners ofthe same type and under different environmental conditions) and dependson the desired tradeoff between False Accept Rate and False Reject Ratebecause σ_(w) ² is a tradeoff parameter that controls the relationshipbetween them. When performing such optimization is not feasible, as avery approximate guideline, herewith we disclose the possible ranges forσ_(w) ² that we determined for the swipe capacitive scanners ofAuthenTec. For the case of direct signal modes of the PreprocessingModule and of the Postprocessing Module, σ_(w) ² can be chosen fromabout 2 to about 10, with very good results achieved for σ_(w) ² equalto about 3; for σ_(w) ² larger than about 15, the performance becomescloser to the one when the Low-pass Implementation of the FilteringModule is used. For the case of inverse signal mode of the PreprocessingModule or of the Postprocessing Module, σ_(w) ² can be chosen from about3·10⁻⁹ to about 3·10⁻⁸, with very good results achieved for σ_(w) ²equal to about 1·10⁻⁸; for σ_(w) ² larger than about 1·10⁻⁷, theperformance becomes closer to the one when the Low-pass Implementationof the Filtering Module is used. Generally, the performance changeslittle when σ_(w) ² varies within the ranges disclosed herein.

D.4.4.3.5 Computing the Smoothed Signal

For each pixel with index k, where k is from 1 to N, each element of thesmoothed signal vector ν^((s)) is computed by:

ν^((s))(k)=ν^((lm))(k)+d(k)·(ν(k)−ν^((lm))(k)).  (71)

Finally, the output signal x of the Filtering Module in thisimplementation is the difference between the input signal ν and thesmoothed signal ν^((s)), corrected with the Wiener mean μ_(w):

x(k)=ν(k)−ν^((s))(k)+μ_(w)  (72)

where k is the current pixel index, an integer from 1 to N. In thepreferred implementation, the Wiener filter mean μ_(w) is set to 0, butother values of μ_(w) are also possible as μ_(w) can be used tocompensate in case when a fixed-valued offset is present so that theoutput signal ν becomes zero mean.

D.4.5 Masking Module

The Masking Module is shown as block 507 in FIG. 9, and has as input thesignal x, and as output the signal y, which is a one-dimensional signalof the same size as x. It has two implementations: Bypass Implementationand Magnitude Masking Implementation.

D.4.5.1 Bypass Implementation of the Masking Module

The output signal y in this implementation is constant 1 for, i.e.,y(j)=1 for all j, and means that all pixels of x will be used in thefurther processing.

D.4.5.2 Magnitude Masking Implementation of the Masking Module

The output signal y in this implementation is constructed by takingthose pixels from the input signal x that have magnitudes smaller thanor equal to a predetermined value (criterion) θ and marking the rest ofpixels as not useful:

$\begin{matrix}{{y(j)} = \left\{ \begin{matrix}1 & {{{if}{{x(j)}}} \leq \theta} \\0 & {otherwise}\end{matrix} \right.} & (73)\end{matrix}$

When y(j)=1 in Equation (73), the corresponding pixel will be used infurther processing; otherwise, when y(j)=0 the pixel will not be used.The predetermined value θ is chosen as to exclude from furtherprocessing two groups of pixels. The first group includes those pixelsclose to the beginning and the end of signal x as they may haveunacceptably inaccurate values as result of artifacts due todiscontinuities associated with the processing of a finite-lengthsignal. Although the techniques disclosed herein such as the replica andconstant padding and the computation shortening significantly mitigatesuch artifacts, sometimes they cannot reduce them sufficiently, which inturn results into a very inaccurate matching score; therefore, excludingsuch pixels is necessary. Twice the length of the filter used in theFiltering Module can serve as a loose upper bound for the total numberof pixels that this group may have. The second group of pixels includespixels whose magnitudes are too large, because we observed that theyprovide a very inaccurate estimate of the scanner pattern due to variousfactors. The number of pixels in this group, masked as not to be used,is typically small (about several). Therefore, if the combined number ofpixels from both groups becomes very large (in the order of tens), it isrecommended that this be analyzed; a possible solution may be increasingthe predetermined value θ, although the problem may also be due to otherreasons.

The optimal value of θ is determined by experimentation and tests. Whenperforming such optimization is not feasible, as a very approximateguideline, herewith we disclose the possible ranges for θ that wedetermined for the swipe capacitive scanners of AuthenTec. For the caseof direct signal modes of the Preprocessing Module and of thePostprocessing Module, θ can be chosen from about 2.5 to about 5, withvery good results achieved for θ equal to about 3.5. For the case ofinverse signal mode of the Preprocessing Module or of the PostprocessingModule, θ can be chosen from about 1·10⁻⁴ to about 3·10⁻⁴, with verygood results achieved for θ equal to about 2.5·10⁻⁴.

The Magnitude Masking Implementation is the preferred implementation ofthe Masking Module.

D.4.6 Matching Module

The Matching Module is shown as block 508 in FIG. 9. Let x_(e) denotethe output signal of the Filtering Module and y_(e) denote the outputsignal of the Masking Module when the input signal g is an imageacquired during the scanner enrolment and x_(q) denote the output signalof the Filtering Module when the input signal g is an image acquiredduring the scanner verification. Using the signals x_(e), y_(e), x_(q),and y_(q), the Matching Module: (i) selects the common pixel indicesmarked as useful in the signals y_(e) and y_(q), (ii) quantifies thesimilarity between the two signals x_(e) and x_(q) for these commonpixel indices in a score, and (iii) produces a decision via the outputsignal d as to whether the two images have been acquired with the samefingerprint scanner by comparing this score with a threshold. When theoutput signal d takes the value 1, this indicates scanner match; when ittakes value 0, this indicates scanner non-match; and when it takes value(−1), this indicates that a decision on matching/non-matching cannot bemade and a new query image must be acquired. If multiple images areacquired during the scanner enrollment, the Matching Module performs (i)and (ii) for each pair of one enrolled image and the query image, andcomputes the average of all scores, which average is used to perform(iii).

The selection of the common pixel indices marked as useful in thesignals y_(e) and y_(q) produces the signal y_(m) so that:

$\begin{matrix}{{y_{m}(k)} = \left\{ \begin{matrix}1 & {{{{if}\mspace{14mu} {y_{e}(k)}} = {{1\mspace{14mu} {and}\mspace{14mu} {y_{q}(k)}} = 1}}\mspace{14mu}} \\0 & {otherwise}\end{matrix} \right.} & (74)\end{matrix}$

where the index k is an integer running from 1 to N. Let D be the set ofall indices k for which y_(m)(k)=1, and let N_(D) be the number ofelements in this set D.

If N_(D) is less than a predetermined number, the Matching Moduleproduces (−1) as the output signal d, which indicates that the number ofcommon pixel indices is insufficient to compute a reliable similarityscore and to make a decision thereof. In this case, acquiring a newquery image is necessary. This predetermined number has to beestablished experimentally and generally it depends on the number ofsensing elements of the swipe scanner. For the swipe capacitive scannersof AuthenTec which have about 150 sensing elements, we recommend thatthis predetermined number be about 50.

Quantifying the similarity between the two signals x_(e) and x_(q) forthe common pixel indices as computed in the signal y_(m) in a score canbe done with the three implementations that follow; however, otherimplementations are also possible.

D.4.6.1 Normalized Correlation Implementation

First, the norms of the signals x_(e) and x_(q) for the indices in theset D are computed by:

$\begin{matrix}{{{x_{e}}} = \sqrt{\sum\limits_{k \in D}{{x_{e}(k)}}^{2}}} & (75) \\{{{x_{q}}} = {\sqrt{\sum\limits_{k \in D}{{x_{q}(k)}}^{2}}.}} & (76)\end{matrix}$

If any of the norms ∥x_(e)∥ or ∥x_(q)∥ is equal to zero, the MatchingModule produces 0 as the output signal d and does not perform furthercomputations. Otherwise, the similarity score z^((nc)) is computed by:

$\begin{matrix}{z^{({nc})} = {\frac{\sum\limits_{k \in D}{{x_{c}(k)}{x_{q}(k)}}}{{{x_{e}}} \cdot {{x_{q}}}}.}} & (77)\end{matrix}$

The output signal d is then computed by comparing the similarity scorez^((nc)) with a predetermined threshold:

$\begin{matrix}{d = \left\{ \begin{matrix}1 & {{{{if}\mspace{14mu} z^{({nc})}} \geq \tau^{({nc})}}\mspace{14mu}} \\0 & {otherwise}\end{matrix} \right.} & (78)\end{matrix}$

The decision threshold τ^((nc)) is the result of optimization and testswith many images and depends on the desired False Accept Rate and FalseReject Rate. As a very approximate guideline, it is typically in therange from about 0.4 to about 0.6.

D.4.6.2 Correlation Coefficient Implementation

First, the zero-mean signals {tilde over (x)}_(e) and {tilde over(x)}_(q) for the indices k in the set D are computed by:

$\begin{matrix}{{{\overset{\sim}{x}}_{e}(k)} = {{x_{e}(k)} - {\frac{1}{N_{D}}{\sum\limits_{k \in D}{x_{e}(k)}}}}} & (79) \\{{{\overset{\sim}{x}}_{q}(k)} = {{x_{q}(k)} - {\frac{1}{N_{D}}{\sum\limits_{k \in D}{x_{q}(k)}}}}} & (80)\end{matrix}$

where the index k runs through all elements in the set D. The values of{tilde over (x)}_(e)(k) and {tilde over (x)}_(q)(k) for indices k thatdo not belong to the set D can be set to 0 or any other number becausethey will not be used in the computations that follow.

Next, the norms of the signals {tilde over (x)}_(e) and {tilde over(x)}_(q) for the indices k in the set D are computed by:

$\begin{matrix}{{{{\overset{\sim}{x}}_{e}}} = \sqrt{\sum\limits_{k \in D}{{{\overset{\sim}{x}}_{e}(k)}}^{2}}} & (81) \\{{{{\overset{\sim}{x}}_{q}}} = {\sqrt{\sum\limits_{k \in D}{{{\overset{\sim}{x}}_{q}(k)}}^{2}}.}} & (82)\end{matrix}$

If any of the norms ∥{tilde over (x)}_(e)∥ or ∥{tilde over (x)}_(q)∥ areequal to zero, the Matching Module produces 0 as an output signal d anddoes not perform further computations. Otherwise, the similarity scorez^((cc)) is computed by:

$\begin{matrix}{z^{({cc})} = \frac{\sum\limits_{k \in D}{{{\overset{\sim}{x}}_{e}(k)}{{\overset{\sim}{x}}_{q}(k)}}}{{{{\overset{\sim}{x}}_{e}}} \cdot {{{\overset{\sim}{x}}_{q}}}}} & (83)\end{matrix}$

The output signal d is then computed by comparing the similarity scorez^((cc)) with a predetermined threshold:

$\begin{matrix}{d = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} z^{({cc})}} \geq \tau^{({cc})}} \\0 & {otherwise}\end{matrix} \right.} & (84)\end{matrix}$

The decision threshold τ^((cc)) is result of optimization and tests withmany images and depends on the desired False Accept Rate and FalseReject Rate. As a very approximate guideline, it is typically in therange from about 0.4 to about 0.6.

The preferred implementation uses the Correlation CoefficientImplementation.

D.4.6.3 Relative Mean Square Error Implementation

First, the norm of the signal x_(e) for the indices in the set D iscomputed as specified by:

$\begin{matrix}{{{x_{e}}}{\sqrt{\sum\limits_{k \in D}\; {{x_{e}(k)}}^{2}}.}} & (85)\end{matrix}$

If the norm ∥x_(e)∥ is equal to zero, the Matching Module produces 0 asoutput signal d and does not perform further computations. Otherwise,the similarity score z^((rmse)) is computed by:

$\begin{matrix}{z^{({rmse})} = {\frac{\sqrt{\sum\limits_{k \in D}\left\lbrack {{x_{e}(k)} - {x_{q}(k)}} \right\rbrack^{2}}}{{x_{e}}}.}} & (86)\end{matrix}$

The output signal d is then computed by comparing the similarity scorez^((rmse)) with a predetermined threshold:

$\begin{matrix}{d = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} z^{({rmse})}} \leq \tau^{({rmse})}} \\0 & {otherwise}\end{matrix} \right.} & (87)\end{matrix}$

The decision threshold τ^((rmse)) is result of optimization and testswith many images and depends on the desired False Accept Rate and FalseReject Rate. As a very approximate guideline, it is typically in therange from about 0.8 to about 1.1.

D.4.6.4 Combined Score

In order to improve the robustness of the score and the accuracy of thescanner authentication, it is recommended to compute several scores(instead of only one) between the signal x_(e) and signal x_(q) (alongwith their corresponding y_(e) and y_(q)) and then combine these scoresinto a single, combined score that is compared with a predeterminedthreshold to determine the scanner match/nonmatch decision. Thus,instead of computing a single score z (for example, by using Equation(77), (83), or (86)) by using all N pixels of the signals x_(e) andx_(q) (and their corresponding y_(e) and y_(q)), the signals x_(e) andx_(q) (and their corresponding y_(e) and y_(q)) can be split into twoparts and the corresponding two scores be computed separately. Forexample, a score z′ can be computed (for example, by using Equation(77), (83), or (86)) for the pixels in x_(e) and x_(q) (and theircorresponding y_(e) and Y_(q)) with the index k running from 1 to └N/2┘and another score z″ can be computed (for example, by using Equation(77), (83), or (86)) for the pixels in x_(e) and x_(q) (and theircorresponding y_(e) and y_(q)) with the index k running from ┌N/2┐ to N.Then the two scores can be combined into one combined score. Using thequadratic mean (also known as a root mean square) to combine the twoscores provides very good results, but other types of means can also beused. When using the quadratic mean, the combined score is computed by:

$\begin{matrix}{z_{combined} = {\sqrt{\frac{\left( z^{\prime} \right)^{2} + \left( z^{''} \right)^{2}}{2}}.}} & (88)\end{matrix}$

The reason to use a combined score is because for the swipe capacitivescanners of AuthenTec, scores z′ and z″ computed as disclosed betweentwo images acquired with one and the same scanner sometimes and for somescanners can substantially differ: one of the scores can be much largerthan 0.5, while the other one—much smaller and even close to 0.Combining them by using a quadratic mean ensures that the combined scoreis sufficiently large so as to result in scanner match decision, thusreducing the probability of false reject. The quadratic mean alsoreduces the probability of false accept. Finally, it is not necessary tosplit the signals in even parts, although the even split provides verygood results.

It is also possible to split the signal x_(c) and signal x_(q) (alongwith their corresponding y_(e) and y_(q)) into more than two parts(e.g., into G parts) and similarly compute G scores. Then, if thequadratic mean is used to combine them, the combined score is:

$\begin{matrix}{z_{combined} = {\sqrt{\frac{\left( z^{\prime} \right)^{2} + \left( z^{''} \right)^{2} + {\cdots \mspace{14mu} \left( z^{G} \right)^{2}}}{G}}.}} & (89)\end{matrix}$

Using a large number G, however, is discouraged as the scores z′, z″,etc., may become unreliable because the number of pixels used in theircomputation may become too small, which in turn may lead to increase(instead of decrease) of the overall accuracy and worsen the errorrates.

Once the combined score is computed, it is compared with thepredetermined threshold that corresponds to the types of scores ascomputed, for example, by Equation (77), (83), or (86). However, whenusing the combined score, it may be necessary to adjust thepredetermined threshold in order to achieve the required error rates.

D.4.7 Using Multiple Images

All exemplary implementations described in Section D.4 are capable ofusing a single image for the scanner enrolment and a single image forthe scanner authentication, and this is preferred because (a) itrequires the least number of computations and (b) it is the most secureas it determines if two images are taken with the same scanner or notwithout any additional images. However, variations are also possible.For example, it is typical for the biometric systems to capture threeimages and use them for enrolling the biometric information. Similarly,another exemplary implementation allows using multiple images for thescanner enrolment and/or multiple images for the scanner verification.This may improve the overall accuracy of the scanner authentication.

In general, the methods for processing multiple images are as previouslydisclosed. Herein, we disclose several exemplary illustrativeimplementations.

Let the number of enrolled images be E and the output signals of theFiltering Module and the Masking Module, when the enrolled image withindex r is being processed, be x_(r) and y_(r), respectively. In thepreferred exemplary implementation, the similarity scores for each pairconsisting of one enrolled image and the query image are averaged andthe resulting average similarity score is used to produce a decision.Thus, if the similarity score between the query image and the enrolledimage with index r is denoted by z_(r) which is computed using Equation(77), (83), or (86), then the average similarity score z_(α) is:

$\begin{matrix}{z_{a} = {\frac{1}{E}{\sum\limits_{r = 1}^{ɛ}\; {z_{r}.}}}} & (90)\end{matrix}$

Finally, the output signal d of the Matching Module is computed usingEquation (78), (84), or (87), depending on which implementation of theMatching Module is used for computing the similarity scores z_(r).

Another implementation computes an “average” enrolled scanner patternfrom all enrolled images and uses this “average” enrolled scannerpattern in the Matching Module. First, the “average” scanner pattern iscomputed by:

$\begin{matrix}{{x_{a}(k)} = {\frac{1}{E}{\sum\limits_{r = 1}^{E}\; {x_{r}(k)}}}} & (91)\end{matrix}$

where k is an integer running from 1 through N. Next, x_(a) is usedinstead of the signal x_(e). The performance of this implementation maybe suboptimal in certain cases because the “average” signal x_(a) maybecome considerably distorted for some pixels and this may result indecision errors.

D.4.8 Image Cropping

Some swipe scanners produce images that contain rows and/or columns ofpixels with constant values around the area of the actual fingerprintpattern. Since the pixels with these constant values carry noinformation about the scanner pattern (and about the fingerprint patterneither), these pixels have to be detected and removed from the imagebefore it is processed further, i.e., the image has to be cropped. Theseareas of constant values typically surround the area of the fingerprintpattern as rectangular pads on the top and bottom and on the left andright of the fingerprint pattern. Finding if a row falls within such apad can be done by computing the absolute values of the differencesbetween each two adjacent pixels along this particular row. If themaximum of these absolute values is 0, this indicates that all pixelvalues in the row are constant and thus this row of pixels has to beremoved. Finding if a column falls within one of these pads can be donein a same way. This test is applied to each row and each column in theimage and the detected rows and columns are removed. The image croppingcan be done before or after the Preprocessing Module.

D.4.9 Implementations and Performance

All implementations of the {Preprocessing Module, Averaging Module,Postprocessing Module, Filtering Module, Masking Module, MatchingModule} can be used in combination with any of the implementations ofthe modules that precede this current module in the conceptual signalflow diagram depicted in FIG. 9. However, different combinations ofmodule implementations may provide different overall performance.

One very well performing, yet very simple, exemplary illustrativenon-limiting combination of module implementations is shown in FIG. 11.The flowchart 452 discloses the signal processing of this implementationusing a single enrolled image g_(e), acquired and processed during thescanner enrolment, and a single query image g_(q), acquired andprocessed during the scanner verification. Although g_(e) and g_(q) areprocessed at different times, the consecutive processing steps areidentical, and therefore herein they are discussed simultaneously. g_(e)and g_(q) are first processed by the Preprocessing Module 502 operatingin its direct signal mode. Its output signals u_(e) and u_(q) areprocessed by the Averaging Module 504. Its output signals p_(e) andp_(q) are processed by the Postprocessing Module operating in its directsignal mode. Its output signals ν_(e) and ν_(q) are processed by theFiltering Module 506 using a moving-average filter. The Masking Moduleoperates in its bypass mode. Finally, the Matching Module computes thecorrelation coefficient and produces the signal d, based on which adecision for scanner match or scanner nonmatch is made.

The disclosed herein signal processing modules can be implemented in theexemplary system 200 shown in FIG. 5 entirely by software programming ofthe processor 202, entirely in hardware, or some modules by softwareprogramming of the processor 202 and some modules in hardware. Forexample, the Averaging Module and the Matching Module can be implementedin the digital hardware 208, while the Preprocessing Module, thePostprocessing Module, and the Filtering Module can be implemented insoftware that runs on one or more processors 202.

FIG. 18 shows the normalized histograms (integrating 1) of thecorrelation coefficients and the scanner authentication decisions of theexemplary implementation shown in FIG. 11 when the query image has beenacquired with the authentic scanner and when the query image has beenacquired with an unauthentic scanner. The tested images have beenacquired with 27 swipe capacitive scanners of AuthenTec, taken from all10 fingers of 2 individuals, with 10 images per finger (5,400 images intotal). Only a single image was used for scanner enrolment and a singleimage for scanner verification, and each image was matched against allother images. With the exemplary decision threshold of 0.56 (also shownin FIG. 18), the empirical Equal Error Rate (i.e., when the False AcceptRate is equal to the False Reject Rate) is about 1·10⁻³.

The performance (as accuracy) of the exemplary implementations we justdescribed earlier is not the best one possible the methods disclosedherein can deliver; rather, it is just an example for their potential.The methods should be considered as a set of tools for achieving thepurpose of scanner authentication. Therefore, the modules and theirmodes as to be implemented in a particular target application should bechosen and their parameters optimized once the specific applicationrequirements and constraints are defined.

The Averaging Method is computationally very efficient. The main reasonfor this efficiency is the one-dimensional signal processing and itssimplicity (for example, it is not necessary to compute evenconventional convolution). The second reason is the absence of anytransforms from one domain into another (like the Fourier transform).These two reasons result in a linear dependence between the number ofcomputations needed and the number of pixels used. Each module hasdifferent modes of operation, allowing granularity with varying degreesof complexity depending on the computational and time constraints.

Another advantage of the disclosed methods is their robustness under awide variety of conditions. All modules and modes of the disclosedmethods are unconditionally stable as there are no feedback loops in anyform and at any level. Due to their simplicity, the disclosed methodscan tolerate round-off effects due to finite precision limitations inparameter, coefficients and signal quantization. This enablesimplementing them in systems, using microprocessors and/or dedicatedcomputational hardware, with fixed-point arithmetic. All computationsrevolve around multiplication and accumulation of two signal samples anddo not involve transforms from one domain to another (transforms areoften susceptible to numerical problems). The signal inversion mode may,however, requires care in implementing it in fixed-point systems.Finally, the disclosed methods have an edge over even future methodsthat require floating-point computations.

As the methods do not require changes in the fingerprint scanners, theycan be implemented in systems that have already been manufactured andeven sold to customers by software and/or programmable hardwareupgrades. Furthermore the implementations of the methods disclosedherein do not incur additional material and manufacturing costs.

D.5 Applications and Advantages of the Exemplary Implementations

The herein described exemplary implementations provide methods forbipartite authentication which comprises biometric authentication (of auser) and biometric scanner authentication. The scanner authenticationuses methods for computing a verification decision about the biometricscanner used to obtain the user biometric data, based on the extractedbiometric scanner pattern.

The fingerprint scanner 110 that is legitimately constructivelyconnected to system 130 in FIG. 2, system 132 in FIG. 3, and system 134in FIG. 4, and that is used for the biometric enrolment of thelegitimate user for the purpose of biometric authentication ishereinafter referred to as an authentic fingerprint scanner. Any otherfingerprint scanner is hereinafter referred to as unauthenticfingerprint scanner. A digital image acquisition of a fingertip that isperformed with the authentic fingerprint scanner is hereinafter referredto as authentic fingerprint image acquisition. Any other digital imageacquisition of the same fingertip that is performed with an unauthenticfingerprint scanner is hereinafter referred to as unauthenticfingerprint image acquisition.

D.5.1 Estimating the Scanner Pattern

One exemplary implementation is directed to a method for estimating thescanner pattern. Estimating the scanner pattern comprises: (1) applyingonce or multiple times an object to the sensor of the fingerprintscanner; (2) acquiring at least one digital image, which step ishereinafter referred to as image acquisition; (3) processing the pixelvalues to extract and subsequently encode a sequence of numberscontaining sufficient information to represent the fingerprint scanner,the sequence of numbers is hereinafter referred to as scanner pattern;and (4) storing the scanner pattern for future use.

In one exemplary implementation of the scanner pattern estimationmethod, the scanner pattern is estimated from a digital image or digitalimages acquired with a predetermined object applied to the fingerprintscanner. The predetermined object is chosen such that the digital imageacquired with an ideal (perfect) fingerprint scanner when thispredetermined object is applied to the scanner would give a digitalimage with uniform (i.e., constant) values for all pixels in the image.The preferred predetermined object depends on the specific sensingtechnology of the particular fingerprint scanner. For capacitivefingerprint scanners, the preferred predetermined object is air. Otherpredetermined objects for capacitive fingerprint scanners include aliquid (e.g., water) and a solid object with a predetermined dielectricconstant; however, other predetermined objects can also be used.

D.5.2 Scanner Authentication Using the Scanner Pattern

Another exemplary implementation is directed to a method for detectingunauthentic fingerprint scanners and unauthentic fingerprint imageacquisitions by using the scanner pattern. This method is hereinafterreferred to as scanner authentication and includes enrolling theauthentic fingerprint scanner and verifying the authenticity of thefingerprint scanner. Enrolling the authentic fingerprint scanner,hereinafter referred to as scanner enrolment, includes: (1) acquiringwith the authentic fingerprint scanner at least one digital image; (2)estimating the scanner pattern from the at least one digital image usingthe methods for scanner pattern estimation disclosed above; and (3)storing the template scanner features into the system for futurereference.

Verifying the authenticity of a fingerprint scanner, hereinafterreferred to as scanner verification, includes: (1) acquiring with thefingerprint scanner at least one digital image; (2) estimating thescanner pattern from the at least one digital image using the methodsfor scanner pattern estimation disclosed above; (3) comparing the queryscanner features with the template scanner features to compute asimilarity score between the query scanner features and the templatescanner features; and (5) converting the similarity score into adecision that the two sets of scanner patterns either do or do not arisefrom the same fingerprint scanner. The decision is hereinafter referredto as scanner match if the similarity score is within a predeterminedrange of values, and the decision is referred to as scanner non-match ifthe similarity score is outside the predetermined range of values. Whenthe decision is scanner match, then the digital image is considered asbeing acquired with the authentic fingerprint scanner, and theacquisition is an authentic fingerprint image acquisition. When thedecision is scanner non-match, the digital image is considered as beingacquired with an unauthentic fingerprint scanner, and the acquisition isan unauthentic fingerprint image acquisition.

The listing of query scanner features and template scanner features willpartially agree and partially disagree depending on whether or not theyoriginated from the same fingerprint scanner, and this will be capturedin the similarity score. It is also possible that the two lists ofscanner features differ in the number of features they contain, in whichcase only the common entries are used in computing the similarity score.

D.5.3 Bipartite Enrollment

Another exemplary implementation is directed to a method for enrollingthe template biometric features and the template scanner features. Thismethod is hereinafter referred to as bipartite enrolment, and isillustrated by flow chart 40 of FIG. 12. Bipartite enrolment comprises:(1) acquiring at least one digital image with the authentic fingerprintscanner in acquiring step 50; (2) enrolling the template biometricfeatures in enrolling step 70; and (3) enrolling the template scannerfeatures in enrolling step 60.

In the preferred exemplary implementation of bipartite enrolment threedigital images are acquired, but acquiring one, two, or more than threedigital images is also possible. In the preferred exemplaryimplementation of bipartite enrolment, both the biometric enrolment andthe scanner enrolment use the same acquired image or the same set ofacquired images.

In another exemplary implementation of bipartite enrolment, the scannerenrolment uses another acquired image or another set of acquired imagesthan the image or images acquired for the biometric enrolment. Thescanner enrolment is performed with a predetermined object applied tothe fingerprint scanner. It is best to acquire the image or the set ofimages used for the scanner enrolment after acquiring the image orimages used for the biometric enrolment. It is also possible to acquirethe image or the set of images used for the scanner enrolment beforeacquiring the image or images used for the biometric enrolment.

D.5.4 Bipartite Verification

Another exemplary implementation is directed to a method for verifyingthe query biometric features and the query scanner features. This methodis hereinafter referred to as bipartite verification. The preferredexemplary implementation for bipartite verification is shown by flowchart 42 in FIG. 13. Bipartite verification comprises: (1) acquiring atleast one digital image with the authentic fingerprint scanner inacquiring step 50, if the quality of the at least one digital image isnot satisfactory, one or more additional digital images may be acquireduntil digital images with satisfactory quality are acquired to replacethe unsatisfactory images; (2) performing biometric verification step72; (3) performing scanner verification step 62 if the decision inbiometric verification step 72 is biometric match; (4) ending withbipartite verification match in step 80 if the decision of scannerverification step 62 is scanner match; ending the bipartite verificationwith bipartite verification non-match step 82 if the decision of thebiometric verification step 72 is biometric non-match ending thebipartite verification with bipartite verification non-match step 82 ifthe decision of the scanner verification step 62 is scanner non-match.

Another exemplary implementation for bipartite verification is shown byflowchart 44 in FIG. 14. This exemplary implementation includes: (1)acquiring at least one digital image with the authentic fingerprintscanner in acquiring step 50, if the quality of the at least one digitalimage is not satisfactory, one or more additional digital images may beacquired until digital images with satisfactory quality are acquired toreplace the images with unsatisfactory quality; (2) performing scannerverification step 64; (3) performing biometric verification step 74 ifthe decision of scanner verification step 64 is scanner match; (4)ending with bipartite verification match step 80 if the decision ofbiometric verification step 74 is biometric match; (5) ending withbipartite verification non-match step 82 if the decision of scannerverification step 64 is scanner non-match; and (6) ending with bipartiteverification non-match step 82 if the decision of biometric verificationstep 74 is biometric non-match.

In the preferred exemplary implementation of the bipartite verificationmethod, both the biometric verification and the scanner verification useone and the same acquired digital image.

In another exemplary implementation of the bipartite verificationmethod, the scanner verification uses another acquired image than theimage acquired for the biometric verification. In this exemplaryimplementation, the scanner verification uses a digital image acquiredwith a predetermined object applied to the fingerprint scanner. It isbest to acquire the image used for the scanner verification afteracquiring the image used for the biometric verification. It is alsopossible to acquire the image used for the scanner verification beforeacquiring the image used for the biometric verification.

Depending on the object used for the scanner enrolment and for thescanner verification, the bipartite authentication provides differentlevels of security. Possible exemplary implementations of the bipartiteauthentication method and the corresponding levels of security each ofthem provides are shown as rows in the table of FIG. 15. A fingerprintscanner's pattern can be estimated from two types of images depending onthe type of the object applied to the fingerprint scanner:

-   -   1. A predetermined, known a priori, object. Since the object is        known, the differences (in the general sense, not only limited        to subtraction) between the image acquired with the        predetermined object and the theoretical image that would be        acquired if the fingerprint scanner were ideal reveal the        scanner pattern because the image does not contain a fingerprint        pattern.    -   2. A fingertip of a person that, generally, is not known a        priori. The acquired image in this case is a composition of the        fingerprint pattern, the scanner pattern, and the scanner noise.

In the exemplary implementation designated as “Scenario A” in FIG. 15,both the scanner enrolment and the scanner verification use an image orimages acquired with a predetermined object applied to the fingerprintscanner. Thus, the images used for the biometric enrolment and for thescanner enrolment are different. The images used for the biometricverification and for the scanner verification are also different. Hence,this implementation provides weak security, but its implementation issimple and it is useful in applications that do not require a high levelof security. The security level of this exemplary implementation can beincreased by other methods.

In the exemplary implementation designated as “Scenario B” in FIG. 15,the scanner enrolment uses an image or images acquired with apredetermined object applied to the fingerprint scanner. Thus, theimages used for the biometric enrolment and for the scanner enrolmentare different. The images used for the biometric verification and forthe scanner verification, however, are the same. For the scannerverification, the scanner pattern is estimated from a digital image ordigital images acquired with the fingertip of the user applied to thefingerprint scanner. This exemplary implementation provides a mediumlevel of security, and it is useful in applications that do not requirea high level of security. The security level of this exemplaryimplementation can be increased by other methods.

In the exemplary implementation designated as “Scenario C” in FIG. 15,the images used for the biometric enrolment and for the scannerenrolment are the same, and the scanner enrolment uses a digital imageor digital images acquired with the fingertip of the user applied to thefingerprint scanner. The images used for the biometric verification andfor the scanner verification are also the same, and the scannerverification uses a digital image or digital images acquired with thefingertip of the user applied to the fingerprint scanner. This exemplaryimplementation provides strong security. This is the preferred exemplaryimplementation.

D.4.5 Exemplary Implementations for Improved Security of User BiometricAuthentication

The method for bipartite authentication can be used to improve thebiometric authentication of a user to a system by detecting attacks onthe fingerprint scanner that replace a digital image containing alegitimate user's fingerprint pattern and acquired with the authenticfingerprint scanner by a digital image that still contains thefingerprint pattern of the legitimate user but has been acquired with anunauthentic fingerprint scanner. This type of attack will become animportant security threat as the widespread use of the biometrictechnologies makes the biometric information essentially publiclyavailable. In particular, since the biometric information has a lowlevel of secrecy, an attacker may possess complete information about thefingerprint of the legitimate user, which includes:

-   -   (a) possession of digital images of the fingerprint of the        legitimate user acquired with an unauthentic fingerprint        scanner, including images acquired in nearly ideal conditions        and with very high resolution;    -   (b) possession of any complete or partial information about        user's fingertip obtained from a latent fingerprint, i.e., from        an impression left by user's fingertip on a surface;    -   (c) possession of fingerprint features (e.g., minutiae)        extracted from user's fingerprint image;    -   (d) ability to artificially produce digital images that are        synthesized from partial or complete information about user's        fingerprint.

An attacker who has full physical access to the network 120 and system130, 132, or 134 in FIGS. 2-4, may replace the digital image acquired bythe authentic fingerprint scanner by another, unauthentic digital image(i.e., acquired with an unauthentic fingerprint scanner or artificiallysynthesized) that contains the fingerprint pattern of the legitimateuser. In this case, the biometric verification will output a biometricmatch (with high probability). Thus, without additional verification,the unauthentic digital image would be accepted as legitimate and wouldlead to positive biometric authentication. By using the method ofbipartite authentication, however, the system will additionally performscanner verification which will determine (with high probability) thatthe unauthentic digital image has not been acquired with the authenticfingerprint scanner, and thus, the bipartite verification will end withbipartite verification non-match (with high probability). Therefore, theattack on the fingerprint scanner will be detected.

A very significant advantage of each exemplary implementation is that itcan be implemented in systems that have already been manufactured andeven sold to customers by upgrading their system software, firmware,and/or hardware (if using programmable hardware blocks), which can bedone even online. The methods and apparatus taught in the prior art foridentifying devices by designing special hardware, in particular analogand/or digital circuits, typically incur material and manufacturing costand are not applicable to systems (including fingerprint scanners) thathave already been manufactured.

Since scanner authentication is essentially only one part of the wholeauthentication process (see bipartite authentication above), objectiveand subjective time constraints are usually in place for such scannerauthentication. Furthermore, the conventional fingerprint verificationalgorithms typically are very computationally intensive. This problemcan be particularly severe in portable devices. Therefore, the scannerauthentication should impose as little additional computational burdenas possible. Although the time requirements for the scanner enrolmentcan be loose (i.e., users would tolerate longer time to enroll theirbiometrics and devices), the scanner verification should take verylittle time, such as one second or even much less. As a consequence,this computational efficiency is a key element of the exemplaryimplementations, leading to straight forward and computationallyefficient implementations.

The exemplary methods and apparatuses disclosed are suited for anysystem that uses biometric authentication using fingerprints, especiallyfor systems that operate in uncontrolled (i.e., without humansupervision) environments. The methods are particularly suited forportable devices, such as PDAs, cell phones, smart phones, multimediaphones, wireless handheld devices, and generally any mobile devices,including laptops, netbooks, etc., because these devices can be easilystolen, which gives an attacker physical access to them and theopportunity to interfere with the information flow between thefingerprint scanner and the system. For example, an attacker may be ableto replace the digital image that is acquired with the authenticfingerprint scanner with another digital image, even of the legitimateuser, but acquired with unauthentic fingerprint scanner, in order toinfluence the operation of the authentication algorithms that arerunning in the system. This possibility exists even in systems that havetrusted computing functionality (e.g., equipped with a Trusted PlatformModule, TPM, that provides complete control over the software, runningin the system) since the attacker needs not modify the software in orderto achieve successful authentication; only replacement of the digitalimage may be sufficient. However, the method and apparatus for bipartiteauthentication, disclosed herein, provides a mechanism to determine theauthenticity of the fingerprint scanner with which the digital image hasbeen acquired and thus detect such an attack.

Another application is in the hardware tokens. Many companies andorganizations provide hardware tokens to their customers or employeesfor user authentication and for digital signing of their transactions,usually by using challenge-response security protocols over a network.Typically, the customers authenticate themselves to the hardware tokenusing a PIN code, a password, and/or a bank card. If the hardware tokenis also equipped with a fingerprint scanner, the methods provided in theexemplary implementations can increase the security of the hardwaretokens by adding authentication using user's fingerprint and detectingattacks on the fingerprint scanner, including replacing the digitalimages acquired with the authentic fingerprint scanner. It is alsopossible to replace the authentication based on a secret code (a PINcode or a password) with biometric authentication (user's fingerprintpattern).

Thus, the methods provided in the exemplary implementations can be usedin bank applications, in mobile commerce, for access to health careanywhere and at any time, for access to medical records, etc.

While the foregoing written description of the invention enables one ofordinary skill in the art to make and use what is considered presentlyto be the preferred implementation or best mode, those of ordinary skillwill understand and appreciate the existence of variations,combinations, and equivalents of the specific implementation methods andapparatuses, and examples described herein. The invention shouldtherefore not be limited by the above described implementations,including methods, apparatuses, and examples, but by all suchimplementations within the scope and spirit of the appended claims.

While the technology herein has been described in connection withexemplary illustrative non-limiting implementations, the invention isnot to be limited by the disclosure. For example, while exemplaryillustrative non-limiting implementations have been described inconnection with self contained biometric scanners, any sort of biometricscanner capable of being connected to a wired and/or wireless networkmay be used. Although exemplary illustrative non-limitingimplementations have been described in connection with the use offingerprint scanners other types of biometric scanners could be usedinstead.

What is claimed:
 1. A method for authenticating a device having anelectronic processing circuit and memory and a fingerprint area scanner,said method comprising: using an electronic processing circuitconfigured to perform the following, (a) acquiring and storing in amemory at least one enrolled image for comparison to at least one queryimage subsequently input to said scanner; (b) decomposing said at leastone enrolled image and said at least one query image using wavelets to,respectively, compute enrolled wavelet coefficients and query waveletcoefficients; (c) performing wavelet reconstruction by, respectively,setting to zero at least the LL-subband coefficients of said enrolledwavelet coefficients to compute enrolled residuals and at least theLL-subband coefficients of said query wavelet coefficients to computequery residuals; (d) masking selected pixels of said enrolled residualsand said query residuals; (e) computing a similarity score betweencommon pixels of said selected pixels from said enrolled residuals andsaid query residuals; (f) comparing said similarity score with athreshold value to determine whether said at least one query imageinputted to the scanner was acquired by the same scanner that acquiredsaid at least one enrolled image; and (g) authenticating said devicewhen it is determined that said at least one query image and said atleast one enrolled image were acquired by the same scanner.
 2. Themethod of claim 1 wherein said at least one enrolled image and said atleast one query image represent biometric information provided by twodifferent individuals.
 3. The method of claim 1 wherein said maskinguses at least one predetermined criterion for determining said selectedpixels.
 4. The method of claim 1 wherein said wavelets are any one ofbiorthogonal wavelets, Daubechies wavelets, and symlets wavelets.
 5. Themethod of claim 1 wherein said computing is any one of computing acorrelation coefficient, computing a normalized correlation, andcomputing a relative mean square error.
 6. The method of claim 1 furthercomprising any one of inverting the pixel values of said at least oneenrolled image and the pixel values of said at least one query imageafter (a) and inverting the pixel values of said enrolled residuals andthe pixel values of said query residuals after (c).
 7. A method foridentifying a device having an electronic processing circuit and memoryand a fingerprint area scanner, said method comprising: using anelectronic processing circuit configured to perform the following, (a)acquiring, storing in memory and processing at least one enrolled imagefor comparison to at least one query image subsequently input to saidscanner; (b) using wavelets of said at least one enrolled and queryimage to compute wavelet coefficients; (c) performing waveletreconstruction by setting to zero at least the LL-subband coefficientsof said wavelet coefficients to compute enrolled residuals and queryresiduals; (d) masking selected pixels of said enrolled residuals andsaid query residuals; (e) computing a sequence of numbers from saidenrolled and query residuals that contains sets of information whichrespectively represents the scanner that acquired said at least oneenrolled image and the scanner to which said at least one query imagewas input; (f) comparing said sets of information to determine whetherthe same scanner acquired both the enrolled image and the input queryimage; and (g) authenticating said device when it is determined that thesame scanner acquired both the at least one enrolled image and the atleast one query image.
 8. The method of claim 7 wherein said maskinguses at least one predetermined criterion for determining said selectedpixels.
 9. The method of claim 7 wherein said wavelets are any one ofbiorthogonal wavelets, Daubechies wavelets, and symlets wavelets. 10.The method of claim 7 wherein said computing is any one of computing acorrelation coefficient, computing a normalized correlation, andcomputing a relative mean square error.
 11. The method of claim 7further comprising any one of inverting the pixel values of said atleast one enrolled image and the pixel values of said at least one queryimage after (a) and inverting the pixel values of said enrolledresiduals and the pixel values of said query residuals after (c).
 12. Asystem for authenticating a device with an electronic processing circuitand memory and a fingerprint area scanner, said system comprising: meansfor acquiring and storing in memory at least one enrolled image forcomparison to at least one query image subsequently input to saidscanner; means for decomposing said at least one enrolled image and saidat least one query image using wavelets to, respectively, computeenrolled wavelet coefficients and query wavelet coefficients; means forperforming wavelet reconstruction by, respectively, setting to zero atleast the LL-subband coefficients of said enrolled wavelet coefficientsto compute enrolled residuals and at least the LL-subband coefficientsof said query wavelet coefficients to compute query residuals; means formasking selected pixels of said enrolled residuals and said queryresiduals; means for computing a similarity score between common pixelsof said selected pixels from said enrolled residuals and said queryresiduals; and means for comparing said similarity score with athreshold value to determine whether said at least one query imageinputted to the scanner was acquired by the same scanner that acquiredsaid at least one enrolled image, wherein said device is authenticatedwhen it is determined that said at least one query image and said atleast one enrolled image were acquired by the same scanner.
 13. Thesystem of claim 12 wherein said at least one enrolled image and said atleast one query image represent biometric information provided by twodifferent individuals.
 14. The system of claim 12 wherein said means formasking uses at least one predetermined criterion for determining saidselected pixels.
 15. The system of claim 12 wherein said means fordecomposing uses wavelets of any one of biorthogonal wavelets,Daubechies wavelets, and symlets wavelets.
 16. The system of claim 12wherein said means for computing computes any one of computing acorrelation coefficient, computing a normalized correlation, andcomputing a relative mean square error.
 17. The system of claim 12further comprising at least one of a means for inverting the pixelvalues of said at least one enrolled image and said at least one queryimage and a means for inverting the pixel values of said enrolledresiduals and the pixel values of said query residuals.
 18. A system foridentifying a device with an electronic processing circuit and memoryand a fingerprint area scanner, said system comprising: means foracquiring, storing in memory and processing at least one enrolled imagefor comparison to at least one query image subsequently input to saidscanner; means for decomposing said at least one enrolled and queryimage using wavelets to compute wavelet coefficients; means forperforming wavelet reconstruction by setting to zero at least theLL-subband coefficients of said wavelet coefficients to compute enrolledresiduals and query residuals; means for masking selected pixels of saidenrolled query residuals; means for: (i) computing a sequence of numbersfrom said enrolled and query residuals that contains sets of informationwhich respectively represents the scanner that acquired said at leastone enrolled image and the scanner to which said at least one queryimage was input; and (ii) comparing said sets of information todetermine whether the same scanner acquired both the enrolled image andthe query image, wherein said device is authenticated when it isdetermined that the same scanner acquired both the at least one enrolledimage and the at least one query image.
 19. The system of claim 18wherein said means for masking uses at least one predetermined criterionfor determining said selected pixels.
 20. The system of claim 18 whereinsaid means for decomposing uses wavelets of any one of biorthogonalwavelets, Daubechies wavelets, and symlets wavelets.
 21. The system ofclaim 18 wherein said means for computing computes any one of computinga correlation coefficient, computing a normalized correlation, andcomputing a relative mean square error.
 22. The system of claim 18further comprising at least one of a means for inverting the pixelvalues of said at least one image and a means for inverting the pixelvalues of said residuals.
 23. The method of claim 1 wherein said devicecomprises a hardware token.
 24. The method of claim 1 wherein saiddevice comprises a PDA.
 25. The method of claim 1 wherein said devicecomprises a computer.
 26. The method of claim 1 wherein said devicecomprises a mobile telephone.
 27. The method of claim 7 wherein saiddevice comprises a hardware token.
 28. The method of claim 7 whereinsaid device comprises a PDA.
 29. The method of claim 7 wherein saiddevice comprises a computer.
 30. The method of claim 7 wherein saiddevice comprises a mobile telephone.
 31. The system of claim 12 whereinsaid device comprises a hardware token.
 32. The system of claim 12wherein said device comprises a PDA.
 33. The system of claim 12 whereinsaid device comprises a computer.
 34. The system of claim 12 whereinsaid device comprises a mobile telephone.
 35. The system of claim 18wherein said device comprises a hardware token.
 36. The system of claim18 wherein said device comprises a PDA.
 37. The system of claim 18wherein said device comprises a computer.
 38. The system of claim 18wherein said device comprises a mobile telephone.